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1.  INTRODUCTION 

The  goal  of  tiie  helicopter  drive  system  condition-monitoring  component  of  the  Air  Vehicle  Diagnos¬ 
tic  System  (AVDS)  program  is  to  develop  technology  that  will  facilitate  transition  to  “condition-based 
maintenance”  (CBM)  for  tiiese  components.  The  specific  system  requirements  as  stated  in  the  AVDS 
Execution  Plan  are  as  follows: 

•  Detect  and  classify  gearbox  faults  in  real  time  in-flight. 

•  Cope  with  variations  between  gearboxes. 

•  Perform  diagnostics  in  a  variety  of  flight  regimes. 

•  Reduce  false-alaim  rates. 

•  Reduce  maintenance  costs  and  facilitate  transition  to  CBM. 

This  report  describes  tiie  performance  of  a  system  developed  by  NRaD  tiiat  meets  these  require¬ 
ments. 
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2  DATA 


2.1  CH-46  HELICOPTER 

The  focus  of  this  project,  because  of  its  age  and  cost  of  operation,  was  on  the  CH-46  helicopter. 
The  power  train  consists  of  two  turbines  that  drive  the  forward  and  aft  transmissions  and  subsequent 
rotors.  Engine  outputs  are  combined  in  a  mix  box.  The  power  train  is  a  constant-  speed,  variable- 
torque  system.  The  effective  torque  varies  as  a  function  of  the  load  on  the  rotor  blades.  Operation¬ 
ally,  the  aft-main  transmission  and  mix  box  assembly  have  been  the  most  troublesome.  Because  of 
these  difficulties,  this  assembly  has  also  been  the  subject  of  significant  false  removals,  that  is,  prema¬ 
ture  aircraft  removal  and  rebuild  (NADEP  Cherry  Point  estimates  30%  of  rebuilds  are  on  aft-main 
transmission/mix  box  assemblies  that  are  defect-free). 

Instrumentation  consisted  of  eight  accelerometers  and  an  optical  tachometer.  The  accelerometers 
are  uni-axial  and  attached  to  modified  case  nuts  symmetrically  positioned  about  the  center-line  of 
the  assembly.  The  optical  tachometer  was  substituted  for  the  rotor  position  indicator.  The  tachometer 
produces  256  pulses  per  shaft  revolution.  The  rotor  position  indicator  shaft  rotates  at  a  nominal  1 14.4 
Hz  that  dictates  a  data  sampling  rate  of  at  least  58.5  kHz. 

2.2  TEST  STAND  DATA 

The  data  set  consists  of  vibration  data  recorded  from  the  aft  transmission  and  mix  box  installed  on  a 
laboratory  test  stand.  The  data  collection  utilized  a  single  transmission  and  mix  box.  Data  were  col¬ 
lected  for  two  defect-free  reassemblies  of  the  mechanical  system.  Additionally,  eight  different 
faulted  components  were  installed  one  at  a  time.  Only  one  faulted  component  was  present  in  the  as¬ 
sembly  for  any  given  test.  Data  were  recorded  at  nine  torque  levels  on  an  International  Recording 
Instruments  Group  B  (IRIG-B)  28-channel  tape  recorder.  The  data  were  then  transcribed  into  the 
digital  domain  using  high-order,  linear-phase  anti-aliasing  filters  and  a  multi-channel,  simultaneous 
sampling/digitization  system.  The  following  faulted  components  were  installed  and  vibration  data 
collected; 

•  Planetary  Gear  (Corrosion  Spalling)* 

•  Spiral  Bevel  Input  Pinion  Bearing  (Corrosion)* 

•  Spiral  Bevel  Input  Pinion  Gear  (Tooth  Spalling)* 

•  Helical  Input  Pinion  Gear  (Chipping)* 

•  Clutch  Bearing  (Brinelling)* 

•  Helical  Idler  Gear  (Crack  Propagation) 

•  Collector  Gear  (Crack  Propagation) 

•  Quill  Shaft  (Crack  Propagation) 

•  Component  fault  naturally  occurring.  Two  components  at  different  defect  levels  utilized. 

Components  provided  by  NADEP  Cherry  Point. 
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Table  1  is  a  comprehensive  list  of  transmission/mix  box  data  that  are  known  to  have  at  least  one  20- 
second,  repeatably  locatable  (via  time-code),  contiguous  block  during  wdiich  all  nine  channels  are  good 
(eight  accelerometer  and  one  tachometer).  The  data  blocks  have  been  examined  for  quality  assurance 
using  both  automated  and  manual  techniques.  In  order  to  locate  such  data  blocks  it  was  necessary  to 
scroll  through  the  entire  analog  recording  cycle  that  corresponded  to  a  valid  torque  level  and  for  which 
ground-truth  was  known,  using  overlapping  data  blocks.  In  cases  where  no  data  were  indicated,  an  er¬ 
ror-free  block  of  20  seconds  could  not  be  found  m  the  original  analog  data  record. 


Table  1.  Transmission/mix  box  test  stand  data  summary. 


27%T 

30%T 

40%T 

45%T 

50%T 

55%T 

60%T 

70%T 

75%T 

80%T 

100%T 

FAULT 

LEVEL 

No  Defect,  First  Test 

X 

N/A 

Planetary  Pinion  Beating  Corrosion 

X 

2 

S.B.  Input  Pinion  Bearing  Corrosion 

X 

X 

X 

X 

X 

X 

X 

X 

X 

1,2 

S.B.  Input  Pinion  Gear  Scuffing 

X 

X 

X 

X 

X 

X 

X 

X 

X 

1.2 

M.B.  Input  Pinion  Gear  Chipping 

X 

X 

X 

X 

2 

M.B.  Idler  Gear  Crack 

X 

X 

X 

X  I 

N/A 

Collector  Gear  Crack 

X 

X 

X 

X 

X 

X 

X 

X 

N/A 

Quill  Shaft  Crack 

X 

X 

X 

X 

X 

X 

X 

X 

X 

N/A 

S.B.  Assembly  Galling 

X 

X 

X 

X 

X 

X 

X 

X 

N/A 

No  Defect,  Second  Test 

X 

X 

X 

X 

X 

X 

X 

X 

X 

N/A 

X  —  20  seconds  of  good  vibration  data 


2.3  IN-FLIGHT  DATA  COLLECTION 

Vibration  data  were  collected  from  nine  deployed  Marine  Corps  CH-46E  aircraft.  The  eight  acceler¬ 
ometers,  locations,  modified  case  nut  mounts,  and  tachometer  were  the  same  as  those  used  in  the  test 
stand  data  collection.  A  14-channel  IRIG-B  t^e  recorder  was  used  to  acquire  the  data.  To  observe  the 
effects  of  naturally  occurring  wear  on  the  transmission/mix  box,  two  data  collections  were  made  on 
each  aircraft  at  widely  separated  intervals  (approximately  6  to  10  months  apart).  Since  the  aircraft  were 
deployed,  the  assumption  was  that  die  data  represented  systems  in  good  working  condition  with  no 
defects. 


To  expand  the  utility  of  the  test  stand  data  to  in-flight  conditions,  it  was  necessary  to  acquire  the  in¬ 
flight  data  in  flight  conditions  that  approximated  those  of  tiie  test  stand.  For  these  data  collections,  pi¬ 
lots  flew  the  aircraft  at  a  constant  torque  and  tried  to  maintain  straight  and  level  flight.  The  achieved 
flight  conditions,  corresponding  to  the  nine  test  stand  torques  shown  in  table  2.  Table  3  lists  in-flight 
transmission/mix  box  data  that  are  known  to  have  at  least  one  20-second,  contiguous  block  during 
which  all  nine  data  channels  are  functioning. 

2.4  FLIGHT  TRIALS  DATA 

Flight  trials  of  the  NRaD-developed,  real-time,  in-flight  data  collection  and  diagnostic  system  were 
conducted  on  a  single  CH-46E  aircraft  atNAWC  Patuxent  River,  MD.  The  accelerometers,  locations, 
modified  case  nut  mounts,  and  tachometer  were  the  same  as  previously  used.  For  the  diagnostic  system 
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Table  2.  Test  stand  torques. 


Torque  Level  (%) 

Flight  Condition 

27 

Below  light  on  wheel 

30 

750  ft/min  descent 

40 

80  knots  air  speed  (KNAS) 

45 

60  KNAS 

50 

light  on  wheels 

60 

100  KNAS 

70 

Hover,  in-groimd  effect 

75 

Hover,  out-of-ground  effect 

80 

120  KNAS 

100 

140  KNAS  VNE 

Table  3.  Phase  II  Transmission  and  mix  box  in-flight  data  (all  no  fault). 


27%T  30%T  40%T  45%T  50%T  55%T  60%T  70%T  75%T  80%T  100%T 

BuNo  156457  A 

B 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

BuNo  156420  A 

B 

X 

X 

X 

X 

X 

X 

X 

X 

BuNo  157673  A 

B 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

BuNo  153395  A 

B 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

BuNo  153960  A 

B 

X 

X 

X 

X 

X 

X 

X 

X 

BuNo  156432  A 

X 

B 

X 

X 

X 

X 

X 

X 

X 

X 

X 

BuNo  154829  A 

X 

X 

X 

X 

B 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

BuNo  154815  A 

X 

i  X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

B 

i _ 

X  —  20  seconds  of  good  vibration  data 


to  operate  autonomously,  an  automatic  flight  regime  recognition  capability  was  required.  This  permit¬ 
ted  the  pilot  to  fly  the  aircraft  without  the  constraints  imposed  during  the  in-flight  data  collections.  The 
system  would  automatically  acquire  data  when  the  aircraft  was  in  a  valid  Data  Acquisition  Flight 
Condition  (DAFC),  previously  defined  in  table  2.  Teledyne  Controls  had  the  task  of  developing  this 
capability  and  integrating  it  into  the  in-flight  system.  Teledyne  also  manufactured  the  Health  and  Usage 
Monitoring  System  (HUMS)  that  was  evaluated  during  these  flight  trials. 

The  flight  trials  consisted  of  1 8  dedicated  flights  and  one  non-dedicated  flight.  The  data  collection 
system  operated  during  17  of  the  19  flights  and  collected  a  total  of  22.37  hours  of  transmission/mix 
box  vibration  data. 
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3  SYSTEM  DESCRIPTION 


3.1  NEURAL  NETWORK  PROCESSOR  (N3P)  SYSTEM  DESIGN 

3.1.1  System  Summary 

The  N3P  system  processes,  in  near  real  time,  transmission/mix  box  data  from  CH-46  helicopters 
and  evaluates  the  mechanical  condition  of  the  mechanical  system.  The  in-flight  N3P  system  consists 
of  a  VME  chassis  with  the  capability  to  digitize  10  analog  data  channels  at  a  nominal  sample  rate  of 
117  kHz,  process  the  data,  and  determine  the  transmission  condition  approximately  every  3.1  sec¬ 
onds.  The  system  also  contains  a  data  recording  capability  that  saves  all  data,  results,  and  communi¬ 
cations  to  8-mm  digital  tapes  for  post-flight  analysis. 


Figure  1 .  N3P  System  in  the  VME  chassis. 


3.1.2  Software  Processing  Flow 

The  N3P  system  has  10  processors  that  perform  the  various  system  functions.  Analog  data  are  re¬ 
ceived  by  the  analog-to-digital  (A/D)  subsystem  digitized  synchronously  with  the  transmission  ta¬ 
chometer  through  a  NRaD-designed  Phase-Lock-Loop  (PLL);  see  figure  1 .  The  resultant  digital 
time-series  data  are  retrieved  by  the  Data  Distribution  (DD)  subsystem.  The  data  are  sent  to  the 
Quality  Assurance  (QA)  and  Feature  Extraction  (FE)  subsystem.  The  Feature  Vectors  (outputs  from 
the  Feature  Extraction)  are  sent  to  both  a  set  of  Radial  Basis  Function  Neural  Networks  and  a  set  of 
Auto-Associator  Neural  Networks,  provided  the  time-series  data  pass  the  QA  tests.  Once  the  Neural 
Network  outputs  are  generated,  they  are  fused  together.  The  fused  RBF  and  AA  results  are  then  rated 
for  acceptability.  If  acceptable  they  are  averaged  and  an  answer  code  is  generated.  The  answer  code 
is  then  assembled  into  the  N3P  output  message,  providing  that  valid  Data  Acquisition  Flight  Condi¬ 
tion  (DFAC)  exists  (see  section  3.3).  Figure  2  is  the  system  block  diagram. 
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Figure  2.  AVDS  block  diagram. 


3.1 .2.1  Executive  Controller  (HOST).  The  N3P  process  starts  with  the  Executive  Controller 
(HOST)  execution.  The  Executive  Controller  does  the  bulk  of  the  decision  making  on  how  the  N3P 
processing  is  performed.  The  HOST  also  provides  all  the  system  services  that  the  N3P  requires  to 
load  and  execute  all  of  the  program  threads.  After  all  the  programs  are  loaded  and  executing,  the 
HOST  signals  the  DD  subsystem  to  begin  data  distribution  and  the  A/D  subsystem  to  begin  data  ac¬ 
quisition,  Once  this  is  done,  the  HOST  waits  for  results  to  arrive.  The  HOST  only  communicates 
over  the  VME  subsystem  bus  to  the  other  N3P  processes  (with  the  exception  of  the  I/O  Support  Sub¬ 
system  that  can  receive  a  reset  over  a  serial  interface  and  control  information  over  the  Ethernet). 

When  signaled  that  the  QA/FE  data  products  are  available,  the  HOST  checks  the  QA  communica¬ 
tion  packet  for  errors.  If  no  fatal  errors  are  present,  the  HOST  transfers  the  Feature  Vector  to  the  NN 
Subsystem  for  processing.  The  system  design  achieves  fault  tolerance  through  the  "banks"  of  neural 
networks.  If  a  single  channel  fault  occurs,  the  HOST  determines  the  transmission  condition  from  the 
network  within  the  bank  that  contains  valid  results.  Otherwise,  all  network  outputs  are  fused  to  de¬ 
termine  the  transmission  state.  The  QA/FE  results  are  always  sent  to  the  Memory  Subsystem  and 
bundled  with  the  data  frame  sent  by  the  DD. 

When  signaled  that  the  NN  processing  is  completed,  the  HOST  retrieves  the  results  from  all  the 
separate  neural  networks  and  fuses  them  into  a  single  result  word.  The  NN  results  are  transferred  to 
the  Memory  Subsystem.  An  answer  code  is  also  written  to  the  Memory  Status  Section  for  transmis¬ 
sion  via  the  Communications  process  of  the  I/O  Support  subsystem. 

3.1. 2.2  Data  Distribution  (DD).  After  initialization,  the  DD  Subsystem  monitors  the  A/D  Subsys¬ 
tem  for  the  data  ready  signal.  Data  ready  triggers  the  process  to  begin  assembling  data  and  de¬ 
multiplexing  it  from  the  raw  stream  format  onto  64K  sample  blocks.  Once  the  data  frame  (64K 
samples  x  10  channels)  is  completed,  it  is  sent  to  the  QA/FE  Subsystem  and  the  Memory  Subsystem. 
The  QA/FE  Subsystem  is  notified  that  a  new  data  frame  is  available  and  the  DD  starts  assembly  of 
the  next  data  frame.  Dual  buffers  (A/B)  are  employed  throughout  to  increase  system  throughput. 

3.1 .2.3  Quality  Assurance/Feature  Extraction  (QA/FE).  The  QA/FE  Subsystem  waits  for  the  data 
ready  signal  from  the  DD  then  processes  the  data  frame  to  determine  that  the  data  is  of  sufficient 
quality  for  NN  processing  (no  dropouts,  saturation,  etc.)  After  quality  checks,  the  feature  set  is 
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generated  and  the  HOST  is  signaled  that  processing  is  complete  and  data  products  are  available.  The 
QA/FE  Subsystem  process  then  waits  for  another  data  frame  ready  and  the  processing  repeats. 

3.1. 2.4  Neural  Networks  (NN).  The  NN  Subsystem,  after  receiving  the  signal  that  a  new  Feature 
Vector  has  arrived,  processes  the  features  through  both  a  Radial  Basis  Function  neural  network  sys¬ 
tem  and  an  Auto- Association  neural  network  system.  Each  network  system  consists  of  a  "bank"  of 
eight  neural  networks  that  provide  for  fault  tolerance.  By  eliminating  a  single  channel  from  each  of 
the  individual  networks,  any  one  channel  can  be  faulty  and  there  is  still  a  single  network  running  that 
can  determine  the  transmission  state.  After  processing  is  complete,  the  HOST  is  signaled  and  the  NN 
process  waits  for  another  Feature  Vector  to  arrive. 

3.1 .2.5  I/O  Support.  The  I/O  Support  Subsystem  consists  of  three  basic  processes:  the  Communica¬ 
tions  process,  the  Phase  Lock  Loop  Control  process,  and  the  Data  Storage  process,  which  provide  the 
critical  functions  of  external  interface  support  and  timing. 

The  Communication  process  receives  and  sends  messages  over  a  RS-232  interface.  It  also  calcu¬ 
lates  and  maintains  the  Data  Acquisition  Flight  Condition  (DFAC)  Status  and  queues  for  N3P  An¬ 
swer  processing.  These  processes  supply  the  HOST  generated  answer  under  "valid"  DFAC  condi¬ 
tions  and  NO-ANSWER,  otherwise. 

The  Phase  Lock  Loop  (PLL)  Control  process  responds  to  analog  data  acquisition  and  QA  errors 
and  can  reset  the  external  PLL  hardware  over  a  digital  line.  It  also  has  the  responsibility  of  main¬ 
taining  the  socket  (Ethernet)  interface  with  the  HOST  used  for  I/O  Support  Subsystem  control. 

The  Data  Storage  process  waits  for  the  HOST  to  indicate  that  a  complete  data  packet  (data  frame, 
QA/FE  results  and  NN  results)  is  ready  and  then  records  the  data  packet  to  a  bank  of  three  Exab3^e 
8505  8-mm  digital  data  recorders. 

3.2  NEURAL  NETWORK  PROCESSOR  HARDWARE 

The  VME  enclosure  contains  circuit  cards  that  perform  the  following  functions:  1)  interface  and 
signal  conditioning  for  eight  accelerometers  and  an  optical  tachometer;  2)  flight  regime  recognition 
and  RS-232  communications  between  HUMS  and  the  N3P;  and  3)  the  Neural  Network  Processor 
(N3P).  Software,  developed  under  this  program,  is  implemented  in  the  N3P  hardware. 

The  in-flight  VME  enclosure  is  a  ruggedized  and  electromagnetic  interference  (EMI)-shielded 
ELMA  chassis.  It  has  a  14-slot  backplane  and  a  six-slot  VSB  overlay.  Table  4  shows  a  circuit  card 
power  budget. 

3.2.1  DIGITAL  STORAGE  DEVICES 

The  chassis  has  a  Kingston  Technology  1GB  hard  disk.  Three  8-mm  Exab)d:e  tape  drives  pro¬ 
vide  archival  data  storage  of  eight  accelerometer  and  optical  tachometer  signals  and  eight  inter¬ 
mediate  system  processing  results. 

3.2.2  NEURAL  NETWORK  PROCESSOR  CARDS 

The  following  6U  size  VME  circuit  cards  constitute  the  N3P  system: 

(1)  Force  2CE  Single  Board  Computer 
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(1)  Motorola  MVME  166  with  MVME  7  12-006  Transition  module 

(2)  SKYbolt8146-V 

(1)  Micro  Memory  256MB  memory 
(1)  ICS-1  lOA  Digitizer 


Table  4.  N3P  power  budget. 


Device 

+5  Volts 

+12Volts 

-12  Volts 

1GB  harddisk  (EST). 

2.55 

2.50 

Three  EXABYTE  8505XL  tape 
drives 

5.40 

2.25 

Slot  1 -FORCE  2CE 

4.00 

1.60 

0.20 

Slot  2-MVME  166 

6.50 

1.00 

0.10 

Slot  3-MVME712-006 

0.28 

0.10 

0.15 

Slot4-MVME712-007 

0.28 

0.10 

0.15 

Slot  7-SKYbolt8146-V 

8.40 

Slot  8-SKYbolt8146-V 

6.40 

Slot  10-256Mbyte  Memory 

7.75 

Slot  11 -ICS-1 10A  A/D 

3.00 

0.20 

0.20 

Slot  12-Spare 

Sum  of  currents 

44.56 

7.75 

0.80 

Watts 

222.80 

93.00 

9.60 

Total  Watts 

325.40 

3.3  NEURAL  NETWORK  PROCESSOR/HUMS  INTERFACE 
3.3.1  General  Operation 

The  HUMS  MDAU  communicates  with  the  N3P  via  a  Matrix  I/O  board  in  the  VME  chassis.  The 
board  provides  three  serial  ports  and  processes  aircraft  flight  data  to  determine  valid  Data  Acquisition 
Flight  Conditions  (DAFC)  for  the  N3P.  A  valid  DAFC  is  achieved  vv^en  the  aircraft  is  in  steady-state 
flight  and  is  at  one  of  the  nine  torque  levels  utihzed  in  the  transmission/mix  box  data  collection  phase  of 
the  project  (discussed  in  section  2).  Acceptable  torque  levels  are  shown  in  table  5.  It  is  important  to 
note  that  the  only  flight  data  available  to  the  N3P  are  whether  the  aircraft  is  in  a  valid  DAFC  or  not. 

The  aircraft  could  be  on  the  ground  (27%  torque)  or  at  140  knots  (100%  torque)  and  still  be  in  a  valid 
DAFC. 
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Table  5.  Acceptable  torque  levels  for  valid  Data  Acquisiton  Flight  Conditions  (DAFC). 


Torque  Level  (%) 

Flight  Condition 

Comments 

27 

Below  light  on  wheel 

Normally  transient 

30 

750  ft/min  descent 

40 

80  knots  air  speed  (KNAS) 

Losing  altitude 

45 

60KNAS 

Maintaining  altitude 

50 

Light  on  wheels 

60 

100  KNAS 

70 

Hover,  in-ground  effect 

75 

Hover,  out-of-ground 
effect 

80 

120  KNAS 

100 

140  KNAS  VNE 

Velocity  not  to  exceed 

3.3.2  Regime  Processor 

The  Regime  Processor  communicates  with  the  DMAU  and  tiie  N3P  via  a  serial  asynchronous  full- 
duplex  communication  chaimel  (figure  3).  Each  of  the  three  processors  (MDAU,  Regime  Processor, 
and  N3P)  transmit  unsoHcited  messages  at  a  uniform  rate.  The  receiving  processor  in  each  case  de¬ 
termines  the  start  of  each  message  by  recognizing  the  inter-message  quiet  time  before  each  message. 
No  handshaking  messages  or  control  signals  are  employed. 


Figure  3.  Regime  Processor  serial  link  signal  flow  diagram. 
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The  regime  processor  analyzes  flight  data  associated  with  each  0.25-second  interval  and  determines 
if  the  aircraft  is  in  a  valid  DAFC  for  that  interval.  Determination  of  valid  DAFCs  is  in  accordance  with 
a  predefined  set  of  flight  parameters. 

3.3.3  Regime  Processor  to  N3P  Message  Content 

Each  0.25  seconds  the  Regime  Processor  sends  a  message  (figure  4)  to  the  N3P.  This  message  con¬ 
tains  the  current  Regime  Processor  status  and  the  most  recently  determined  DAFC.  The  date  and  time 
values  are  derived  from  the  MDAU  internal  time-of-day  clock.  The  values  sent  to  the  N3P  are  adjusted 
for  transit  time,  so  that  the  N3P  stores  a  consistent  time  with  minimal  processing.  The  values  in  each 
message  field  are  formatted  in  hexadecimal. 

3.3.4  N3P  to  Regime  Processor  Message  Content 

The  N3P  sends  an  event  message  to  the  regime  processor  once  each  second.  This  message  (figure  5) 
contains  the  current  status  and  the  present  condition  of  the  transmission/mix  box  assembly.  The  values 
in  each  message  field  are  hexadecimal. 

3.3.5  Regime  Processor  to  MDAU 

The  content  of  the  four  most-recent  N3P  event  messages  and  the  current  regime  are  sent  to  the 
MDAU  once  each  second  for  recording  on  the  Optical  Quick  Access  Recorder  (OQAR).  Figure  6 
shows  the  message  format. 
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BYTE# 

CONTENT 

1 

Status 

2 

Spare 

3 

DAFC 

4 

Aircraft  Type 

5 

Aircraft  Tail  ID 

6 

Message  Year 

7 

Message  Month 

8 

Message  Day  of  Month 

9 

Message  Hour 

10 

Message  Minutes 

11 

Message  Seconds 

12 

Checksum 

Figure  4.  Regime  Processor  to  N3P  message  content. 
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BYTE# 

CONTENT 

1 

Status 

2 

Spare 

EBi 

Fault 

5 

Message  Hour 

6 

Message  Minute 

7 

Message  Second 

8 

Message  Year 

9 

Message  Month 

10 

Message  Day  of 

Month 

11 

Event  Hour 

12 

Event  Minute 

13 

Event  Second 

14 

Checksum 

11 

Event  Hour 

12 

Event  Minute 

13 

Event  Second 

14 

Checksum 

Figure  5.  N3P  to  Regime  Processor  message  content. 
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BYT! 

1 

BYTE  2 

BYTE  3 

BYTE  4 

BYT! 

E  5 

BYTE  6 

IS 

EDSI 

■1 

IBS 

IBS 

BBS 

B 

If 

iEBSB 

B 

B 

BBBS 

Bi 

B 

B 

EBB 

BEB 

B 

1 

BBBEBS 

IB 

FAULT 

M.SEC 

■ 

BY1 

rE  7 

BYTE  8 

BY1 

CE  9 

BYTE  10 

IBI 

SB 

11 

im 

BI 

mi 

[BE 

B 

8 

Bf 

fBI 

IBi 

IBB 

B 

B 

EBB 

BI 

EB 

i 

imyjjyiyi 

E.MIN 

in 

:ec 

lyjgigyillig 

BYT] 

E  11 

BYTE  12 

BYTE  13 

BYTE  14 

BYTE  15 

IBEBI 

BEB 

BEB3 

BEBOI 

Bi 

BEBBBB 

1  2  3  4  5  6 

/  8 

1  :  3  (i  5  f  7  8 

DAFC2 

DAFC3 

DAFC4 

FRAME  COUNT 

00 

CKSUM 

Legend: 

STATUS 

STATUS 

08 

bits 

E.  HOUR 

EVENT  HOUR 

05  bits 

FAULT 

FAULT 

16 

bits  ■ 

E.MIN 

EVENT  MINUTE 

06  bits 

M.HOURS 

MESSAGE  HOURS 

05 

bits 

E.  SEC 

EVENT  SECOND 

06  bits 

M.  MIN 

MESSAGE  MINUTE 

06 

bits 

DAFC  1 

Fourth  Most  Recent  Calculated  DAFC 

06  bits 

M.  SEC 

MESSAGE  SECOND 

06 

bits 

DAFC2 

Second  Most  Recent  Calculated  DAFC 

06  bits 

M.  YEAR 

MESSAGE  YEAR 

07 

bits 

DAFC  3 

Third  Most  Recent  Calculated  DAFC 

06  bits 

M.  MON 

MESSAGE  MONTH 

04 

bits 

DAFC  4 

Most  Resent  Calculated  DAFC 

06  bits 

M.  DOM 

MESSAGE  DAY  OF  MONTH 

05 

bits 

00 

Padding  for  byte  alignment 

02  bits 

CKSUM 

CHECKSUM 

08  bits 

Figure  6.  Regime  Processor  to  MDAU  message  content. 
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4  NEURAL  NETWORK  PROCESSOR 
LABORATORY  BENCH  TESTS 


4.1  INTRODUCTION 

The  objectives  of  these  tests  was  to  quantify  the  system  performance  and  determine  if  the  re¬ 
quirements  stated  in  the  AVDS  execution  plan  are  being  met  (discussed  in  section  1).  To  quantify 
the  system  performance,  the  following  metrics  are  defined: 

1.  Probability  of  False  Alarm  (PFA)  -  the  number  of  times  an  alarm  condition  was  issued 
when  the  underlying  ground-truth  indicated  that  a  no  defect  condition  existed. 

2.  Correct  Condition  Identification  (CCI)  -  correct  alarm  condition  (alann/no  alarm)  based  on 
the  ground-truth  of  the  data. 

3.  Correct  Classification  (CC)  -  probability  of  correct  classification  given  that  an  alarm 
condition  existed  according  to  the  ground-truth  of  the  data. 

4.1.1  Probability  of  False  Alarm 

The  PFA  metric  quantifies  the  system’s  potential  for  false  alarm.  A  high  PFA  indicates  that  the 
system  has  a  tendency  to  false  alarm.  A  low  PFA  indicates  that  insofar  as  the  limitations  of  the 
testing  procedures  allow,  the  system  appears  to  be  robust  and  not  subject  to  significant  concerns 
regarding  false-alarm  rates. 

4.1.2  Correction  Condition  Identification  (CCI) 

The  CCI  metric  quantifies  the  system’s  performance  in  correctly  issuing  an  alarm  or  no  alarm 
condition. 

4.1 .2.1  CCIi.  A  high  CCIi  indicates  that  the  system  performs  well  overall  in  correctly  identifying  the 
condition  (alarm/no  alarm)  of  the  transmission.  A  low  CCIi  indicates  that  overall  tiie  system  id  not 
perform  well  in  correctly  identifying  the  condition  of  the  transmission.  Poor  performance  of  the  CCIi 
metric  can  be  attributed  to  four  factors:  false  positives  (false  alarm),  false  negatives,  no  reports 
(including  those  caused  by  poor  data  quality),  and  system  failures. 

4.1 .2.2  CCI2.  A  high  CCI2  indicates  that  when  a  condition  could  be  determined  for  the  transmission, 
the  system  performed  well  in  correctly  identifying  this  condition.  A  low  CCI2  indicates  that  specifically 
when  condition  reports  were  issued,  the  system  did  not  perform  well. 

4.1. 2.3  CCI1  and  CCI2  Together.  A  low  CCIi  with  a  high  CCI2  indicates  that  the  system  performs 
well  when  a  condition  could  be  identified,  but  the  system  is  plagued  by  no  reports  and/or  system  fail¬ 
ures.  Most  likely,  the  no  reports  would  be  due  to  low  neural  network  confidence,  DAFC  transition  in¬ 
stabilities,  and  poor  data  quality. 

A  low  CCIi  with  a  low  CCI2  indicates  that  the  system  performed  poorly  in  identifying  the  condition 
of  the  transmission  ntix  box.  This  also  indicates  that  the  system  made  a  substantial  number  of  false 
positive  or  false  negative  determinations.  If  the  PFA  is  high,  the  system  made  a  significant  number  of 
false  positive  determinations.  Otherwise,  the  system  made  a  significant  number  of  false  negative  de¬ 
terminations. 
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4.1.3  Correct  Classification 

The  CC  metric  quantifies  the  system’s  performance  in  correctly  classifying  the  transmission’s  condi¬ 
tion  to  one  of  a  set  of  predetermined  states  (currently  eight  defect  states  and  one  no-defect  state). 

4.1.3.1  CCi.  A  high  CCi  indicates  that  the  system  performs  well  overall  in  correctly  classifying  the 
health  of  the  transmission  to  a  predetermined  state.  A  low  CCi  indicates  that  overall  the  system  did  not 
perform  well  in  correctly  classifying  the  status  of  the  transmission.  Poor  performance  of  the  CCj  metric 
can  be  attributed  to  five  factors:  misclassification,  novel  conditions  that  do  not  conform  to  one  of  the 
predetermined  states,  poor  data  quahty,  no  reports,  and  system  failures. 

4.1. 3.2  CC2.  A  high  CC2  indicates  that  when  a  condition  could  be  determined  for  tiie  transmission,  the 
system  performed  well  in  further  correctly  classifying  this  condition  to  one  of  a  somev^at  narrow  set  of 
defects  or  no  defect.  A  low  CC2  indicates  that  specifically  when  condition  reports  were  issued,  the 
system  did  not  perform  well  in  further  resolving  the  status  of  the  transmissiorr 

4.1. 3.3  CCi  and  CC2  Together.  A  low  CCi  with  a  high  CC2  indicates  that  the  system  classifies  well 
when  a  condition  could  be  identified,  but  the  system  is  plagued  by  poor  data  quality,  novel  conditions, 
DAFC  transition  instabilities,  or  system  failures. 

A  low  CCi  with  a  low  CC2  mdicates  that  the  system  performed  poorly  in  classifying  the  status  of  the 
transmission.  This  also  mdicates  that  the  system  made  a  substantial  number  of  misclassifications.  If 
the  PFA  is  high,  the  system  made  a  significant  number  of  false  alarm  misclassifications.  If  the  PFA  is 
midrange  to  low,  the  system  made  a  significant  number  of  misclassifications  across  the  set  of  classes. 

4.2  DIGITAL  PERFORMANCE  TEST 

Digital  performance  tests  were  carried  out  using  test  stand  data  only  (section  2.2)  stored  on 
optical  read-write  media.  The  data  records  are  each  approximately  20  seconds  long  and  consist  of 
continuous  digitally  sampled  data  in,  by  definition,  a  valid  Data  Acquisition  Flight  Condition 
(DAFC).  The  N3P  system  requires  0,588  seconds  of  data  to  make  a  call.  Therefore,  34  calls 
should  be  made  over  a  20-second  period*.  These  data  have  previously  passed  both  automated  and 
manual  quality  assurance  checks  and  is  not  known  to  contain  any  sensor  failures  or  artifacts. 

4.2.1  Test  Phase  I  -  Baseline  Digital  Performance  Test 

4.2.1. 1  Test  Format.  System  performance  for  this  phase  of  testing  is  qualified  using  the  ground-truth 
of  the  data.  The  data  records  to  be  used  for  this  phase  of  testing  and  the  sequence  of  testing  are  listed  in 
table  6.  Note  that  the  data  records  corresponding  to  no-fault  data  represent  approximately  30%  of  the 
total  data  with  the  remaining  70%  of  data  comprised  of  fault  data  Also  note  that  each  subset  of  data  is 
represented  by  a  complete  set  of  DAFCs  (data  availability  permitting). 


'  Specifically,  34  calls  would  be  made  if  the  system  had  been  up  and  running  loi^  enough  to  stabilize,  the  DAFC  had  been 
constant  long  enough  to  have  been  judged  a  stable  DAFC  and  enough  prior  data  had  been  accumulated  to  fill  the  integrator, 
etc. 
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4.2.1 .2  Test  Methodology.  The  purpose  of  Phase  I  Testing  was  to  quantify  the  baseline  digital 
performance  of  the  N3P  System.  Phase  I  Testing  will  be  based  upon  the  correct  issuance  of  an 
alarm  or  no  alarm  condition.  An  alarm  condition  is  further  broken  down  to  reflect  correct  classifi¬ 
cation  statistics.  In  addition,  false  alarm  results  are  also  be  collected.  For  the  purposes  of  Phase  I 
Testing,  each  data  frame,  regardless  of  origin,  has  an  equal  weight  in  determining  the  overall  sta¬ 
tistics  of  the  test.  Each  20-second  data  record  is  played  into  the  N3P  System  and  the  results  will 
be  tabulated  in  order  to  quantify  the  performance  of  the  system. 

4.2.1 .3  Test  Criteria.  At  the  highest  level,  each  data  frame  falls  into  one  of  two  classes:  (1)  flie  class 
for  vvhich  an  alarm  condition  should  be  generated  (defect)  and  (2)  die  class  for  which  no  alarm  condi¬ 
tion  should  be  generated  (no  defect).  Each  data  frame  processed  by  the  N3P  System  is  graded  to  de¬ 
termine  the  performance  of  the  system  in  determining  the  correct  condition  (alarm/no  alarm)  based  on 
die  ground-truth  of  the  data.  The  output  of  this  portion  of  Phase  I  Testing  is  the  Correct  Condition 
Identification  (CCI)  statistic.  The  CCI  value  is  determined  by  dividing  the  number  of  data  frames  for 
winch  a  correct  answer  was  issued  by  die  total  number  of  data  frames.  This  metric  is  calculated  using 
two  different  sets  of  data  The  first,  CCIi,  measures  all  data  frames  and  all  N3P  outputs.  The  sec¬ 
ond,  CCI2,  is  limited  to  those  data  frames  for  which  the  N3P  system  reported  a  condition.  Conse¬ 
quently,  a  no  condition  report  is  considered  an  error  by  the  CCIi  metric  but  it  is  ignored  by  the 
CCI2  metric.  The  difference  between  the  two  CCI  values  will  quantify  the  effect  of  startup  and 
transient  on  the  system’s  stability.  Each  data  record  will  be  “cold-started^”  into  the  N3P  system  to 
enhance  repeatabihty. 

In  addition,  the  probabihty  of  correct  classified  (CC)  and  the  probability  of  false  alarm  (PFA)  are 
determined.  The  CC  metric  is  the  probability  of  correct  classification  given  an  existing  alarm  condition 
according  to  the  ground-truth  of  the  data  As  before,  the  CC  metric  will  be  separated  into  two  separate 
statistics.  The  first,  CCi,  measures  all  data  frames  and  all  N3P  outputs  while  the  second,  CC2,  meas- 
mes  only  those  frames  for  which  the  N3P  system  reported  a  condition.  Again,  the  difference  between 
the  two  CCs  will  quantify  the  effect  of  startup  and  transients  on  the  system’s  stability. 

The  probability  of  false  alarm  is  calculated  using  all  of  the  no-defect  data  for  which  a  condition  re¬ 
port  was  issued.  It  is  determined  by  dividing  the  number  of  data  frames  for  which  an  alarm  condition 
was  issued  when  the  imderl5ting  ground-truth  indicated  that  no  defect  condition  existed  by  the  total 
number  of  data  frames  for  which  the  N3P  system  issued  a  response.  Note  that  this  measure  excludes 
those  data  frames  for  which  no  report  was  issued  by  the  N3P  System. 

4.2.2  Test  Phase  II  -  Digitally  Degraded  Sensor  Environment  Performance  Test 

4.2.2.1  Data  Format.  System  performance  for  this  phase  of  testing  is  quantified  using  the  ground 
truth  of  the  data.  In  order  to  simulate  a  degraded  sensor  environment,  a  single  channel  of  each 
record  is  digitally  zeroed  in  software.  This  simulates  the  loss  of  signal  associated  with  a  channel. 
The  data  records  to  be  used  for  this  phase  of  testing,  the  sequence  of  testing,  and  the  data  channel 
to  be  zeroed  for  each  record  are  listed  in  table  7.  Note  that  the  data  records  comprise  the  same 
basic  subset  of  data  used  during  Phase  I  Testing. 


^i.e.,  the  N3P  System  software  is  re-initialized  between  each  data  record. 
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Table  6.  Phase  I  digital  baseline  performance  test. 
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Table  7.  Phase  II  digitally  degraded  sensor  environment  test. 
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4.2.2.2  Test  Methodology.  The  piupose  of  Phase  n  Testing  is  to  quantify  the  performance  and  ro¬ 
bustness  of  the  N3P  System  in  the  presence  of  a  degraded  signal  environment.  Phase  n  Testing  will  be 
based  upon  the  correct  issuance  of  an  alarm  or  no-alaim  condition.  An  alarm  condition  will  be  further 
broken  down  to  reflect  correct  classification  statistics.  In  addition,  false  alarm  results  are  also  collected. 
For  the  purpose  of  Phase  n  Testing,  each  data  frame,  regardless  of  origin,  will  have  equal  weight  in 
determining  the  overall  statistics  of  the  test.  Each  20-second  data  record  is  played  into  the  N3P  System 
and  the  results  will  be  tabulated  in  order  to  quantify  the  performance  of  tiie  system.  As  in  Phase  I,  each 
data  record  will  be  cold-started  into  the  N3P  System. 

4.2.2.3  Test  Criteria.  The  same  criteria  used  in  Phase  I  of  the  testing  is  used  to  determine  system 
performance  during  Phase  II  of  the  testing:  CCI,  PFA,  and  CC. 

4.2.3  Test  Phase  Hi  -  Digital  DAFC  Transition  False  Alarm  Performance  Test 

4.2.3.1  Data  Format.  System  performance  for  this  phase  of  testing  is  quantified  using  the  ground- 
truth  of  the  data.  In  order  to  simulate  r^id  transitions  among  DAFCs,  a  single  data  record  of  approxi¬ 
mately  20  seconds  duration  is  created.  The  data  record  will  consist  of  data  frames  from  randomly  se¬ 
lected  DAFCs  of  a  single  aircraft  (A/C).  The  data  frames  are  combined  into  a  single  data  record  of  34 
frames  to  play  into  the  N3P  System.  The  data  frames  to  be  used  for  this  phase  of  testing  and  the  se¬ 
quence  of  combination  of  the  frames  are  listed  in  table  8. 

4.2.3.2  Test  Methodology.  The  purpose  of  Phase  III  Testing  is  to  quantify  the  false  alarm  rate  of  the 
N3P  System  as  a  function  of  transition  between  DAFCs.  Phase  DI  Testing  is  based  on  upon  the  col¬ 
lection  of  false  alarm  statistics.  For  the  purpose  of  Phase  HI  Testing,  each  data  frame,  regardless  of 
origin,  will  have  equal  weight  in  determining  the  false  alarm  rate.  The  data  record  is  played  into  the 
N3P  System  and  flie  results  will  be  tabulated  in  order  to  quantify  the  performance  of  the  system.  The 
first  frame  of  the  data  record  is  cold-started  into  the  N3P  System. 

4.2.3.3  Test  Criteria.  The  criteria  used  in  Phase  IE  of  the  testing  to  determine  system  performance  is 
PFA 

4.3  DIGITAL  PERFORMANCE  TEST  RESULTS 

Phase  I  and  II  Combined 

Criteria  CCI  CC  PFA 

Value  99.9%  96.0%  0.0% 

Phase  I  -  Baseline  Digital  Performance  Test 

Criteria  CCIi  CCI2  CCi  CC2  PFA 

Value  100.0%  100.0%  96.2%  96.2%  0.0% 


Note:  In  all  cases,  the  condition  of  the  transmission  according  to  ground-truth  was  correctly  identified. 
Cases  where  the  Correct  Classification  was  not  made  occurred  during  the  first  and/or  second  data 
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frame  of  a  record  where  the  Radial  Basis  Fxinction  (RBF)  nets  had  not  yet  stabilized.  There  were  seven 
data  frames  (Hehcal  Idler  Gear  Crack  80%T)  were  the  AAs  incorrectly  identified  the  fault  as  normal.  It 
is  believed  that  this  due  to  the  threshold  of  the  AAs  being  set  to  error  on  the  side  of  no  false  alarm.  The 
breakdown  of  calls  for  each  data  record  used  in  Phase  I  Testing  is  included  in  table  9. 


Phase  II  -  Digitally  Degraded  Sensor  Environment  Performance  Test 


Criteria  CCIi 

Value  99.9% 


CCI2  CCi 

99.9%  95.8% 


CC2  PFA 

95.8%  0.0% 


Note:  All  cases  where  the  correct  condition  of  the  transmission  were  not  identified  were  false  nega¬ 
tives  (i.e.,  a  ground-truth  defect  condition  was  identified  by  the  anomaly  detectors  as  a  good  transmis¬ 
sion)  diat  occurred  at  a  DAFC  of  70%  Torque.  Cases  vvfrere  the  Correct  Classification  was  not  made 
are  once  again  the  first  few  data  frames  of  a  record  where  the  RBF  nets  had  not  yet  stabilized.  There 
were  18  data  frames  (SB  Bearing  Corrosion,  Helical  Idler  Gear  Crack,  and  Quill  Shaft  Crack)  were  the 
AAs  incorrectly  identified  the  fault  as  normal.  Once  again,  this  beheved  to  be  due  to  the  threshold  of 
the  AA’s  being  set  to  err  on  the  side  of  no  false  alarm.  The  breakdown  of  calls  for  each  data  record  is 
included  in  table  10. 


Phase  III  -  Digital  DAFC  Transition  False  Alarm  Performance  Test 

Criteria  PFA  CCI  CC 

Value  0.0%  100.0%  97.0% 

Note:  There  were  no  false  alarms  during  this,  or  any,  portion  of  testing.  While  the  criteria  of  CCI  and  CC 
are  not  performance  criteria  for  this  test  phase  according  to  the  test  plan,  these  criteria  were  both  satisfied 
by  any  saise  of  the  measure.  The  breakdown  of  calls  for  each  data  record  is  included  in  table  1 1 . 
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Table  8.  Phase  III  digital  DAFC  transition  false  alarm  test. 
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Note:  If  an  KRaD  number  appears  more  than  once,  sequential  data 
frames  will  be  used  from  that  data  file.  Otherwise,  the  first  data  file 
wall  be  used. 
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Table  9.  Baseline  digital  performance  test  results  by  data  record. 
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Table  10.  Digitally  degraded  sensor  environment  performance  test  by  data  record. 
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5  DETECTION  OF  A  FAULT  IN  BU  NO.  156432 


5.1  USE  OF  IN-FLIGHT  DATA 

The  transmission/mix  box  diagnostic  neural  networks  were  trained  to  recognize  one  of  eight  test 
stand  conditions  (seven  different  defect  conditions  and  a  no-defect  condition).  These  training  data 
were  limited  to  a  single  mechanical  assembly  that  underwent  several  rebuilds  during  the  seeded  fault 
trials.  Additionally,  the  data  were  from  an  assembly  that  was  installed  in  a  massive  test  stand  and 
driven  by  electric  motors  into  a  water  brake  load.  To  extend  the  diagnostics  to  a  realistic  environ¬ 
ment,  recorded  in-flight  vibration  data  were  used.  Sixty-four  features  indicative  of  the  condition  of  a 
particular  subassembly,  in  the  transmission/mix  box,  were  extracted  from  each  of  the  eight  vibration 
channels.  These  features  were  combined  into  a  single  feature  of  512  elements  that  represented  the 
overall  condition  of  the  transmission/mix  box  for  a  particular  Data  Acqusition  Flight  Condition. 
These  exemplars  were  used  to  augment  the  original  network  training.  This  was  expected  to  increase 
the  robustness  and  reduce  false  alarms  due  to  differences  between  the  test  stand  and  in-flight  envi¬ 
ronments.  The  data,  from  multiple  aircraft,  could  also  be  used  to  evaluate  the  capability  to  perform 
diagnostics  on  a  transmission/mix  box  without  first  establishing  a  baseline  signature.  (The  Teledyne 
HUMS  required  25  in-flight  hours  to  establish  a  baseline  signature).  Implicit  in  the  use  of  in-flight 
data  from  deployed  Marine  Corps  aircraft  is  the  assumption  that  the  data  represent  transmission/mix 
box  assemblies  in  good  working  order  with  no  defects. 

5.2  RADIAL  BASIS  FUNCTION  NEURAL  NETWORKS 

A  neural  network  differs  from  classical  techniques  of  mechanical  diagnosis  in  that  the  network  is 
trained  using  a  representative  sample  of  data  out  of  the  “universe”  of  data  or  features  the  network  is 
expected  to  encounter.  This  training  allows  the  network  to  form  statistical  models  of  the  types  of 
features  that  the  network  will  be  expected  to  recognize.  Essentially,  the  network  builds  its  own  sta¬ 
tistical  defect  models  using  examples  from  the  feature  sets  that  it  is  desired  it  recognize.  Assuming 
that  the  training  feature  sets  are  truly  representative  of  the  particular  classes  of  data,  the  statistical 
models  are  inherently  robust  and  have  an  ability  to  generalize  across  the  entire  class  of  data. 

This  statistical  model  is  then  used  to  recognize  other  examples  of  the  same  t5q>e  of  defect.  Since 
the  models  are  robust,  the  network  can  generalize  to  features  that  are  similar  but  not  exactly  the  same 
as  the  training  sets.  It  is  desired  that  this  robustness  could  be  exploited  so  that  the  network  would  not 
have  to  be  baselined  using  data  from  each  and  every  transmission/mix  box  to  which  they  would  ever 
be  applied.  At  this  time,  the  program  was  using  a  Radial  Basis  Function  (RBF)  network  as  the  major 
technique  of  classification  since  RBF  networks  have  many  characteristics  that  make  them  desirable 
to  use  in  this  application.  Some  of  the  characteristics  that  the  program  was  trying  to  exploit  are  that 
RBFs  work  well  as  classifiers,  that  a  RBF  network  forms  “soft”  as  opposed  to  sharply  delineated  de¬ 
cision  surfaces,  and  that  RBF  nets  can  be  rapidly  trained  on  a  relatively  large  quantity  of  features. 

RBF  neural  networks  have  the  common  network  architecture  of  simple  processing  elements  or 
nodes  that  are  heavily  intercormected  and  are  typically  constructed  as  a  feed-  forward  network  with  a 
single  hidden  layer.  Each  node  of  the  hidden  layer  simultaneously  receives  the  entire  input  vector 
and  is  fully  interconnected  to  an  output  layer  of  linear  units  (see  figure  7).  Each  hidden  layer  node 
uses  a  radial  basis  function  to  form  a  localized  receptive  field  that  calculates  the  "closeness"  of 
the  input  vector  to  a  particular  data  cluster  in  feature  space.  The  closer  the  input  vector  is  to  the  cen¬ 
ter  of  the  cluster  the  more  strongly  the  neuron  responds  with  an  upper  bound  of  one. 
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The  RBF  used  is  typically  a  gaussian  basis  or  a  logistic  function.  This  type  of  function  gives  rise  to 
a  symmetric  receptive  field  in  feature  vector  space  (figure  8).  The  combination  of  these  basis  func¬ 
tions  in  the  hidden  layer  gives  rise  to  "soft"  decision  surfaces  where  there  is  no  hard  boundary  that 
encloses  a  decision  region.  This  soft  decision  surface  allows  the  RBF  net’s  performance  to  degrade 
somewhat  gracefully  as  a  function  of  deterioration  of  data  quality.  As  data  quality  degrades,  the  fea¬ 
tures  extracted  tend  to  fall  further  from  the  centers  of  the  data  clusters,  resulting  in  a  smaller  output 
for  any  given  class  or  combination  of  classes.  This  can  be  contrasted  with  the  hard  decision  surfaces 
formed  by  a  typical  Multi-Layer  Perceptron  (MLP). 


Figure  7.  Typical  architecture  of  a  RBF  neural  network.  This  network  has  a  single  hidden  layer  of 
RBFs  fully  interconnected  to  an  output  layer  of  linear  units.  Each  output  of  an  RBF  node  is  multiplied 
by  an  interconnection  weight  that  is  determined  by  training.  The  weighted  outputs  of  the  RBF  nodes 
are  then  summed  at  the  corresponding  linear  unit.  After  summation,  the  output  of  the  linear  node  is 
determined  using  a  sigmoid  transform  to  bound  the  output. 


30 


1 

0.8 

0.6 

0.4 

0.2 

0 


Output  for 


Figure  8.  Modified  RBF 
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,  where 


in  two-dimensional  feature  space  formed  using  the  modified  logistic  function 
(p  represents  the  bias  added  to  the  receptive  field,  a  represents  the  width  of 


the  field,  and  |x  represents  the  center  of  the  field.  Note  that  the  function  gives  rise  to  a  symmetric, 
localized,  receptive  field. 


Each  hidden  unit  of  a  multi-layer  perception  is  used  to  form  a  hyperplane  as  opposed  to  a  localized 
receptive  field.  This  hyperplane,  in  combination  with  hyperplanes  formed  from  other  hidden  layer 
nodes,  encloses  a  decision  region  of  feature  space.  Typically,  it  takes  four  or  more  hyperplanes  to  en¬ 
close  a  given  decision  region.  The  input  feature  vector  is  either  inside  or  outside  of  this  bounded  deci¬ 
sion  region  as  opposed  to  the  continuum  of  responses  provided  by  a  soft  decision  surface.  This  strictly 
delineated  decision  boundary  is  a  potential  source  of  high  false  positives  for  a  multi-layer  perception 
approach. 

To  form  the  RBF  network,  the  features  extracted  from  the  data  are  first  clustered  in  some  manner  to 
reduce  the  complexity  of  the  problem  and  to  speed  computations.  It  is  desired  that  this  clustering  be 
efficient  without  losing  the  essential  character  of  the  features.  The  best  clustering  is  achieved  when  the 
clusters  densely  populate  the  data-rich  regions  of  the  feature  space.  This  speeds  training  since  most 
training  techniques  only  update  those  weights  associated  with  a  stimulated  cluster  during  any  given 
training  cycle  and  improves  classification  accuracy.  Our  approach,  as  is  common  in  the  neural  network 
community,  has  been  to  use  the  ^-means^algorithm  to  cluster  the  feature  training  set.  The  choice  of  the 
number  of  clusters,  k,  is  arbitrary  and  typically  confirmed  using  cross-valation  techniques.  Due  to  the 
architecture  of  RBF  networks,  these  types  of  nets  perform  very  well  on  features  that  have  well- 
separated  data  clusters. 


’  J.  MacQueen.  1967.  “Some  Methods  for  Classification  and  Analysis  of  Multivariate  Observations,”  In  Proceedings  of  the 
Fifth  Berkeley  Symposium  on  Mathematical  Statistics  and  Probability,  Volume  I,  Statistics.  L.  M.  Le  Cam  and  J.  Neyman,  eds. 
Univeristy  of  California  Press,  Berkeley,  CA. 
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After  the  features  have  been  clustered,  a  RBF  corresponding  to  a  single  middle  layer  neuron  is 
placed  at  the  centroid  of  each  cluster.  By  centering  the  RBF  of  this  neuron  on  the  given  data  cluster,  the 
receptive  field  of  this  neuron  has  been  tuned  to  respond  to  a  feature  vector  input  near  the  given  data 
cluster.  All  of  the  RBF  neurons  are  then  connected  in  turn  to  all  of  the  output  class  neurons  with  each 
output  class  represented  by  a  single  neuron  in  the  output  layer.  This  neuron  is  typically  a  linear  unit  that 
sums  its  interconnection  weighted  inputs,  and  then  applies  a  sigmoid  transform  to  bound  the  output 
between  zero  and  one  (figure  9).  The  network  is  then  trained  to  recognize  a  particular  class  of  data  us¬ 
ing  a  representative  set  of  features  corresponding  to  those  data. 


Input 

Figure  9.  Sigmoid  transfer  function  that  takes  the  sum  of  inputs  to  a  linear  output  neuron  and 
bounds  the  output  response  to  [0,1],  Note  that  the  input  can  vary  from  -oo  to  oo  in  two  dimensions,  but 
the  output  will  always  be  "squashed"  to  between  zero  and  one.  Hence  the  nickname,  squashing 
function. 

Once  the  network  architecture  has  been  determined,  the  network  can  be  trained  and  the  interconnec¬ 
tion  weights  determined.  RBF  networks  are  typically  trained  on  one  class  of  data  at  a  time.  During  the 
training,  representative  features  from  the  class  being  trained  are  presented  to  the  network  and  the  inter¬ 
connection  weights  for  the  RBF  nodes  or  data  clusters  are  adjusted  until  the  network  does  an  adequate 
job  recognizing  this  class  of  data.  The  next  class  of  data  are  then  trained  with  its  associated  weights 
adjusted  and  so  forth,  until  all  classes  of  data  have  been  trained.  The  training  is  insensitive  to  the  se¬ 
quence  of  classes  that  is  presented  to  the  net.  Once  the  network  has  been  fully  trained,  it  is  ready  to  be 
cross-validated  using  hold-out  techniques  and  other  methods. 
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5.3  AUGMENTING  THE  TRAINING  OF  THE  RBF  NETWORK  USING  IN-FLIGHT  DATA 

In  order  to  train  and  evaluate  the  performance  of  the  RBF  network  using  the  in-flight  data,  a  training 
feature  set  and  a  cross-validation  set  of  features  were  required.  Due  to  differences  in  the  method  of 
data  collection  between  the  in-flight  and  test-rig  data  sets,  it  was  necessary  to  design  a  preprocessing 
filter  transform  to  align  tiie  two  data  sets.  This  necessitated  retraining  the  entire  network  since  the 
baseline  data  had  fundamentally  changed.  The  same  test-rig  training  set  as  used  previously  was  reused 
along  with  a  subset  of  the  total  in-flight  data  set. 

During  the  training  process,  it  was  desired  that  the  network  develop  a  c^abilily  to  generalize  and  to 
correctly  classify  as  demonstrated  using  die  cross-valation  techniques.  The  need  to  develop  these  ca¬ 
pabilities  across  a  range  of  torques  and  airframes  drove  the  design  of  the  training  set.  Obviously,  a  rep¬ 
resentative  sample  of  each  defect  was  required  with  a  corresponding  sanqile  of  no-defect  conditions. 
Since  the  no-defect  condition  is  the  nominal  condition  and  was  assumed  to  occur  for  a  large  number  of 
transmissions,  die  amount  of  no-defect  training  data  are  over-represented  relative  to  any  given  fault 
class  but  the  amount  of  no-defect  training  data  were  roughly  the  same  as  the  traiomg  data  for  all  fault 
classes.  It  was  also  desirable  for  each  training  exemplar  to  be  represented  across  die  full  range  of 
torques  available  in  order  to  provide  for  a  robust  classification  capability.  These  concerns  provided  the 
framework  for  the  partitioning  of  die  in-flight  data  into  a  training  set  and  a  cross-valation  set 

Given  the  total  amount  of  in-flight  data  available,  feature  sets  fi'om  four  of  the  A/C  were  chosen  to 
augment  the  training  set  and  feature  sets  fi'om  the  remaining  four  A/C  were  chosen  for  the  evaluation 
set  This  decision  was  based  on  the  amount  of  data  available  from  each  A/C  for  both  the  initial  and  final 
data  collections^,  the  torques  available  for  all  A/C,  and  the  torques  available  for  any  given  A/C.  Note 
diat  no  data  from  the  A/C  chosen  for  the  cross  validation  feature  set  had  ever  been  presented  to  the 
network  during  the  training  process.  This  type  of  “blind”  cross  validation  using  features  from  a  dis- 
tincdy  new  and  separate,  but  similar  type  of  data  diat  the  network  should  be  able  to  correctiy  classify 
based  on  generalization  c^abilities,  is  commonly  referred  to  as  hold-out  testing,  if  done  properly.  It  is 
worth  emphasizing  that  to  be  true  hold-out  testing,  the  data  must  be  uniquely  distinct  from  the  data 
forming  the  training  set  and  original  to  the  network. 

The  training  data  set  was  then  used  to  train  the  RBF  network.  First  the  features  were  clustered  using 
a  modified  k-means  algorithm.  Several  choices  of  the  number  of  centroids  were  evaluated  for  the  pur¬ 
poses  of  cross  vahdation  and  network  complexity  tradeoff  studies.  The  different  networks  were  then 
trained  on  the  same  data  set  using  supervised  learning  techniques.  Each  of  the  seven  defect  classes  of 
training  data  from  the  test-rig  were  sequentially  presented  to  each  network  and  the  interconnection 
weights  adjusted  until  the  network  correctly  classified  the  defect  class  witii  an  arbitrarily  low  Stun- 
Squared  Error  (SSE).  The  combined  no  defect  class  of  training  consisting  of  in-flight  and  test-rig  no¬ 
defect  data  were  then  presented  to  the  networks  in  the  same  manner  with  corresponding  adjustment  to 
the  interconnection  weights.  After  training,  no  further  adjustments  were  made  to  the  networks  and  they 
were  ready  for  cross  validation. 


T>ue  to  instramentation/sensor  malfonaions,  the  initial  data  colleaions  from  five  of  the  A/C  were  unusable  and  all  of 
the  data  from  one  A/ C  proved  unusable. 
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5.4  INDICATION  OF  AN  ANOMALOUS  CONDITION  DURING  CROSS  VALIDATION 

During  the  cross  validation  of  the  trained  networks,  data  from  three  of  the  A/C  were  consistently  and 
confidently  classified  as  corresponding  to  a  no-defect  condition,  as  expected.  The  networks’  response 
to  the  features  extracted  from  the  second  data  collection  on  Bu  No.  1 56432,  however,  proved  to  be 
quite  unexpected.  For  the  two  lowest  torques  at  which  data  were  collected,  the  networks  consistently 
responded  as  if  they  were  confused  as  to  the  condition  of  this  particular  transmission.  In  particular,  the 
response  for  the  network  lying  at  the  knee  of  the  performance  versus  the  complexity  curve  is  shown  in 
figure  10.  Additionally,  the  classification  at  the  higher  torques  was  consistently  no  defect,  but  at  a  de¬ 
graded  confidence  level.  The  response  for  all  networks  trained  was  consistent  and  similar. 


1 

0.9 

0.8 

0.7 

0.6 

0.5 

0.4 

0.3 

0.2 

0.1 


(a) 


T  4'  I 


Output  Class 


1 

0.9 

0.8 

0.7 

0.6 

0.5 

0.4 

0.3 

0.2 

0.1 


(b) 


5 

Output  Class 


1 

0.9 

0.8 

0.7 

0.6 

0.5 

0.4 

0.3 

0.2 

0.1 

0^ 


(C) 


( 

11 

*  $  4  4  ¥ 


5 

Output  Class 


Figure  10.  Typical  Whisker  plots  showing  the  distribution  of  RBF  Output  Node  responses:  (a)  typical 
no-defect  data;  (b)  data  from  Bu  No.  156432  at  27%  torque;  (c)  data  from  Bu  No.  156432  at  30% 
torque.  The  bars  indicate  the  standard  deviation  of  the  distributions  while  the  boxes  contain  75%  of 
the  data  with  the  mean  of  the  distribution  indicated  by  the  horizontal  bars  within  the  box.  For  the  typi¬ 
cal  no-defect  response,  only  the  first  class  or  no-defect  node  is  highly  excited,  with  all  other  nodes 
showing  a  low,  approximately  constant,  level  of  excitation.  For  the  data  from  Bu  No.  156432,  all  node 
responses  are  relatively  low  level,  with  the  node  having  the  highest  response  toggling  as  a  function 
of  torque. 


In  general,  the  RBF  output  nodes  were  showing  a  low-level  response  to  all  classes  previously  trained 
on,  with  no  confident  classification  to  any  particular  class  of  defect  or  no  defect  previously  trained  oa 
Further,  the  node  that  seemed  to  be  most  highly  excited  was  not  stable  as  a  function  of  torque.  For  the 
two  lowest  torques,  the  highest  node  output  toggled  among  classes  four  and  eight.  No  significance  is 
attached  to  the  particular  classes  excited  since  the  network  was  most  likely  responding  to  a  type  of  data 
clustered  outside  of  its  "universe"  and  the  scarcity  of  data  makes  analysis  problematic. 

This  type  of  behavior  from  RBF  classifiers  is  typically  indicative  of  presentation  to  the  network  of  a 
type  of  data  with  which  they  have  no  previous  experience  and  have  not  been  trained  to  recognize.  If  the 
data  have  not  been  corrupted  in  some  manner  (i.e.,  "garbage  in— garbage  out")  and  most  resembles  the 
characteristics  of  a  class  of  data  clusters  on  wdiich  the  network  has  been  trained,  the  output  class  repre¬ 
senting  these  clusters  will  be  the  one  most  highly  excited.  If  the  data  do  not  resemble  any  of  the  charac¬ 
teristics  of  the  data  clusters,  the  response  of  a  typical  RBF  net  is  not  a  priori  determinable  except  in 
certain  pathologic  cases.  The  situation  is  analogous  to  training  a  RBF  network  to  recognize  red  or 
green,  and  then  presenting  the  network  with  a  yellow  and  asking  it  to  tell  you  whether  it  is  red  or  green. 
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In  some  sense,  it  is  kind  of  red,  and  kind  of  green,  but  the  correct  response  is  that  it  is  neitiier  and  is 
novel  to  the  network. 

Given  our  past  e3q>erience  with  these  data,  this  type  of  behavior  was  apical  due  to  the  nature  of  and 
consistency  of  the  misclassification.  Previously,  we  had  found  that  there  was  a  locally  optimal  number 
of  data  clusters  for  which  the  size  of  the  RBF  employed  tended  to  be  irrelevant  (within  reasonable  lim¬ 
its)  to  the  performance  of  the  network.  As  the  number  of  clusters  increased,  performance  did  not  sig¬ 
nificantly  increase  and  tended  to  depend  on  the  size  of  the  RBF  employed.  When  the  number  of  clus¬ 
ters  reached  an  architecture  and  data-dependent  threshold,  adding  more  clusters  decreased  the  per¬ 
formance  independent  of  die  size  of  the  radial  basis  fimction.  As  the  number  of  data  clusters  was  de¬ 
creased  below  the  local  optimum,  performance  was  highly  dependent  on  the  size  of  the  radial  basis 
function  used  and  tended  to  be  degraded  compared  to  the  networks  trained  using  the  locally  optimum 
number  of  clusters.  In  all  cases  when  misclassificadons  were  previously  observed,  they  tended  to  be 
distributed  among  several  to  all  of  the  data  classes  and  inconsistent  within  any  given  class.  Our  experi¬ 
ence  with  the  features  extracted  firom  Bu  No.  156432  was  the  first  time  tihat  we  had  observed  consis¬ 
tent  misclassification  confined  to  a  single  class  for  all  of  the  network  architectures  used  during  the 
cross-valation  procedure. 

5.5  INVESTIGATION  OF  ANOMALY 

Once  die  anomaly  had  been  consistently  indicated,  the  baseline  data  were  once  again  examined  re¬ 
garding  its  integrity.  The  baseline  automated  quality  assurance  algorithms  were  rerun  on  the  raw  data 
widi  no  deficiencies  in  the  data  indicated.  These  algorithms  had  been  developed  to  check  for  sensor 
faults  and  data  deficiencies  that  had  previously  been  seen  in  the  avionics  environment.  The  algorithms 
also  checked  to  ensure  that  the  data  collection  system  was  fimctioning  properly  and  did  not  induce  any 
artifacts  into  the  data. 

The  raw  data  were  tihen  manually  examined  in  minute  detail  to  determine  if  a  new  type  of  sensor 
fault  or  data  collection,  system-induced  problem  had  been  encountered  during  the  collection  of  this 
particular  data  set.  There  was  no  indication  from  visual  inspection  of  the  raw  data  that  data  had  been 
corrupted  in  any  manner.  To  all  intents  and  purposes,  the  raw  time-series  data  appeared  to  be  free 
firom  defects.  The  data  did  have  slightly  different  amplitude  levels  on  two  of  the  eight  channels  (one 
elevated  and  one  decreased)  when  compared  to  the  minimal  amount  of  data  available  from  the  initial 
collection*,  but  the  magnitude  of  the  changes  were  within  normal  deviations  of  all  the  data  examined  to 
date.  The  conclusion  was  drawn  that  the  anomaly  was  not  caused  by  corrupted  data 

The  training  and  cross-valation  data  sets  were  then  repartitioned  in  order  to  further  investigate  the 
anomaly.  Each  of  the  three  cross-valation  set  members  that  did  not  behave  in  an  anomalous  manner 
was  sequentially  switched  with  a  single  member  of  the  initial  training  data  set.  Three  RBF  networks 
having  a  different  number  of  k-means  clusters  and  different  cluster  widths  were  then  trained  using  the 
new  training  data  set.  The  networks  were  then  evaluated  using  the  new  cross-valation  data  set.  This 
investigation  continued  until  all  original  members  of  the  training  data  set  had  been  replaced  in  turn  by 
one  of  the  members  of  the  original  cross-valation  set  that  did  not  display  anomalous  behavior.  In  total, 


*  Due  10  instrumentation  difficulties,  only  one  complete  data  record  of  approximately  20  seconds  duration  was  available 
from  the  first  data  coUeaion.  The  anomaly  may  or  may  not  have  been  present  at  this  time  but  the  short  duration  of  the  data 
record  did  not  allow  enhanced  signal  averages  to  be  formed  and  very  little  analysis  or  comparison  could  be  done  with  such  a 
small  amount  of  data. 
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27  different  networks  were  trained®  and  evaluated.  In  all  cases,  the  behavior  of  the  networks  still  indi¬ 
cated  that  the  data  from  the  low  torques  of  Bu  No.  156432  were  different  from  the  remainder  of  the  in¬ 
flight  data  and  the  test-rig  no-fault  data,  and  that  this  difference  was  statistically  significant. 

Lastly,  a  generic  feed-forward  MLP  network  with  two  hidden  layers  was  selectively  trained  u<;iTig 
the  baseline  training  data  set  and  standard  back  propagation  techniques  of  training.  The  back  propaga¬ 
tion  algorithm  was  slightly  modified  so  that  during  each  training  epoch,  the  cross-valation  data  were 
also  presented  to  die  network  and  a  SSE-computed  for  the  data  from  each  of  the  A/C.  Although  the 
cross-valation  data  were  presented  during  the  training  process,  no  adjustment  of  the  interconnection 
weights  was  made  for  the  cross-valation  data  The  errors  due  to  the  cross-valation  data  set  were  not 
allowed  to  back  propagate  tiirough  the  net,  only  the  errors  due  to  learning  to  recognize  the  baseline 
training  data  set  were  allowed  to  back  propagate  and  cause  weight  adjustments.  As  tihe  network 
As  the  network  converged,  die  SSEs  for  all  of  the  A/C  from  the  cross-valation  data  set  significantly 
decreased  while  the  MSE  calculated  for  the  data  from  Bu  No.  1 56432  stayed  relatively  large  and 
roughly  constant.  This  trend  in  the  SSEs  is  shown  in  figure  1 1 . 

5.6  NOTIFICATION  OF  NAVAL  AVIATION  SYSTEMS  COMMAND  (NAVAIR)  REGARDING  THE 
ANOMALY 

At  this  time,  the  decision  was  made  to  notify  NAVAIR  that  an  anomaly  had  occurred  during  the 
processing  of  the  in-flight  data  and  that  the  anomaly  was  indicative  of  a  transmission  that  differed  from 
the  seven  other  transmissions  available  for  comparison.  Naval  Air  Warfare  Center  Aircraft  Division 
(NAWCAD)  PAX  River  Flight  Test  Engineering  Group  and  NAVAIR  4.4.2  were  contacted  and  in¬ 
formed  that  the  data  from  A/C  BuNo  1 56432  was  anomalous  when  compared  to  the  data  from  the 
seven  other  A/C  in  our  database?.  After  discussions  including  NAVAIR,  diagnosticians  at  the  Original 
Eqmpment  Manufacturer  (OEM)  and  NAWCAD  Trenton,  the  decision  was  made  to  down  the  A/C  and 
notify  the  associated  squadron  of  the  downing  due  to  a  potential  aft  transmission  failure. 

At  the  time  that  the  A/C  was  downed,  the  time  on  tins  particular  aft  transmission  was  830.6  hours. 
The  Time  Between  Overhaul  (TBO)  for  this  fype  of  transmission  is  900  hours.  Typically,  a  10%  life 
waiver  is  requested,  and  the  aft  transmission  will  fly  for  approximately  990  hours  between  overhauls. 
The  data  collection  causing  the  anomalous  response  was  started  at  an  aft  transmission  time  of  710.7 
hours  and  continued  for  approximately  20  hours  at  Marine  Corps  Air  Station  El  Toro  where  the  A/C 
had  been  attached  to  Marine  Medium  Helicopter  Squadron  (HMM)  764.  Recent  to  the  downing,  the 
A/C  had  been  deployed  ferrying  troops  ship-to-shore  during  Operation  Kemal  Blitz  ‘95. 

After  downing,  the  transmission  was  to  be  removed  and  sent  to  Naval  Depot  (NADEP)  Cherry  Point 
for  Engineering  Inspection  (El)  to  determine  the  status  of  the  transmission  components.  If  the  A/C  had 
not  been  downed,  it  could  have  been  expected  to  fly  for  another  1 59.4  hours  since  there  were  no  other 


*  Note  that  for  each  network  trained,  the  centers  of  the  initial  data  clusters  differed.  Further,  the  weights  for  each  training 
session  were  randomly  initialized  prior  to  convergence.  Thus,  none  of  the  networks  after  training  were  idPHTifal 

^  The  Flight  Test  Engineering  Group  at  Pax  River  had  been  responsible  for  the  data  coUeaion  while  NAVAIR  4.2.2  had 
program  oversight  regarding  the  aft  transmission  neural  network  effort. 
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Figure  1 1 .  Trend  in  the  SSE  as  a  function  of  training  epoch.  The  SSEs  have  been  normalized  by  the 
maximum  sum-squared  error  encountered  during  the  training  run.  The  SSE  for  the  training  data  set 
is  plotted  as  a  solid  line.  The  SSE  for  the  three  A/C  in  the  cross-valation  data  set  that  did  not  have  an 
anomalous  response  are  shown  as  the  lightly  shaded  lines  while  BuNo  156432  is  shown  as  the 
darkly  shaded  line. 

indications  of  a  potential  problem  with  the  transmission.  Periodic  Spectral  Oil  Analysis  Program 
(SOAP)  bums  had  been  passed  with  no  anomalies  indicated,  and  the  squadron  reported  no  special 
maintenance  actions  for  ^s  A/C.  Additionally,  no  unusual  behavior  from  the  transmission  had  been 
reported  by  the  crew.  To  the  date  of  downing,  the  only  indication  that  there  was  a  potential  problem 
with  the  transmission  had  been  the  anomalous  response  of  the  RBF  networks. 

5.7  RESULTS  OF  THE  ENGINEERING  INSPECTION  (El) 

The  El  on  transmission  S/N  A9-328  was  performed  by  the  NADEP  Cherry  Point  personnel  who  are 
responsible  for  performing  the  standard  900-hour  rebuilds  on  the  CH-46  transmission.  When  the 
transmission  cases  were  cracked,  separating  mix  box  from  the  aft  transmission,  the  spiral  bevel  ring 
and  pinion  gear  were  found  to  have  obvious  and  considerable  damage.  Both  mating  surfaces  of  the 
gears  had  considerable  wear  to  the  point  of  cold  metal  working  on  their  meshing  surfaces.  The  wear 
was  evenly  distributed  for  all  teeth  of  the  gears  and  a  considerable  portion  of  the  mesh  was  worn 
(please  refer  to  figures  12  and  13).  The  wear  was  so  extensive  that  depot  personnel  called  it  severe 
galling.  Depot  personnel  also  said  that  if  this  was  not  the  worst,  then  it  was  the  second-worst  type  of 
this  damage  that  they  had  seen. 

As  the  El  progressed,  the  collector  gear  was  found  to  be  in  good  condition  with  a  minimal  and  nor¬ 
mal  amount  of  wear.  Intervening  bearings  and  shafts  were  also  found  to  be  in  good  condition  and  at 
proper  preload  and  alignment.  No  other  components  were  found  to  be  damaged  until  the  helical  input 
pinions  were  removed  from  the  mix  box  case.  When  the  input  pinions  were  removed,  the  starboard 
input  pinion  was  found  to  have  a  localized  chip  or  gouge  and  associated  wear  surrounding  the  gouge 
(figure  14).  It  is  believed  that  this  gouge  was  caused  by  debris  passing  through  the  mesh. 
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To  summarize  the  El,  the  transmission  was  anomalous  and  damaged.  Two  out  of  the  three  areas  we 
had  identified  as  potential  problems  were  found  to  have  defects.  The  defects  were  non-trivial  and 
caused  rejection  of  the  parts  by  NADEP  Cherry  Point.  In  particular,  the  spiral  bevel  gear  had  a  very 
high  degree  of  wear  and  was  new  at  the  time  that  this  transmission  was  last  rebuilt.  Such  gears  typi¬ 
cally  last  two  to  three  rebuilds. 

5.8  AUTO-ASSOCIATOR  NEURAL  NETWORK 

Once  die  transmission  had  proved  defective,  the  in-flight  defect  data  were  used  to  validate  and  dem¬ 
onstrate  the  utility  of  a  new  type  of  neural  network.  We  had  recently  been  working  with  an  Auto- 
Associator  (AA)  or  encoding  network  for  novelty  detection.  This  type  of  network  is  a  three-layer,  feed¬ 
forward  network  that  attempts  to  map  the  output  onto  the  input  using  a  reduced  number  of  hidden  or 
middle  layer  nodes  (figure  1 5).  In  this  ty^ie  of  architecture,  the  input  layer  and  output  layer  have  the 
same  number  of  nodes  in  order  to  facilitate  quantifying  the  success  in  moping  the  ou^ut  onto  the  in¬ 
put.  The  middle  layer  has  a  reduced  number  of  nodes  vriien  compared  to  the  input  and  output  layers. 
The  purpose  of  the  hidden  layer  is  to  encode  the  essential  character  of  the  training  class  by  use  of  a  re¬ 
duced  set  of  information  about  the  training  class. 


(a)  (b) 


Figure  12.  Damage  to  spiral  bevel  ring  gear  (a)  general  and  (b)  close-up.  Damage  occurs  around 
the  entire  gear  and  across  a  significant  portion  of  the  mesh.  Galling  has  worn  through  the  black 
metal  oxide  coating  and  is  close  to  cold  metal  working. 
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Figure  13.  Damage  to  spiral  bevel  input  pinion  gear  (a)  general  and  (b)  close-up.  The  damage  is 
symmetric  to  that  of  the  mating  spiral  bevel  ring  gear.  Again,  the  galling  has  worn  through  the  black 
metal  oxide  coating  and  is  close  to  cold  metal  working. 
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Figure  15.  Architecture  of  a  typical  AA  network.  This  network  has  three  layers  that  are  fully  intercon¬ 
nected.  The  hidden  or  middle  layer  is  of  a  reduced  number  of  nodes  when  compared  to  the  input  or 
output  layer  in  order  to  encode  the  essence  of  the  training  class  of  data.  The  input  and  output  layer 
have  the  same  number  of  nodes.  After  training,  the  output  of  a  feature  vector  is  compared  to  the  in¬ 
put  and  the  RMS  error  is  calculated  between  the  two.  The  error  is  then  compared  to  a  threshold.  If 
the  error  is  greater  than  the  threshold,  the  input  is  considered  novel. 


This  type  of  network  differs  significantly  from  typical  pattern  classifiers  in  that  only  "positive"  data 
corresponding  to  a  single  class  of  data  is  used  to  train  the  network.  During  training,  the  network  learns 
to  map  the  output  onto  the  input  by  encoding  the  essential  character  of  the  training  class  into  the  hidden 
layer.  This  is  done  by  taking  the  output  vector,  reconstructed  from  the  hidden  layer,  and  comparing  it  to 
the  input  vector.  A  Root-Mean-Squared  (RMS)  error  is  then  calculated  from  the  difference  of  the  input 
and  output  vectors.  The  interconnection  weights  are  adjusted  during  the  training  process  to  reduce  the 
RMS  prediction  error. 

Once  the  network  has  been  trained  and  a  sufficiently  small  RMS  prediction  error  achieved  for  the 
elements  of  the  training  data  set,  a  threshold  is  set  on  the  RMS  error  using  cross-valation  techniques. 
Typically,  the  threshold  is  set  so  that  all  members  of  the  training  class  of  data  fall  below  the  threshold 
while  RMS  prediction  errors  calculated  from  vectors  corresponding  to  other  classes  of  cross-valation 
data  fall  above  the  threshold.  The  network  is  then  ready  to  be  used  in  a  specific  application.  When  the 
network  is  employed,  a  feature  vector  never  seen  before  by  the  network  is  passed  in  as  input.  The 
middle  layer  attempts  to  reconstruct  the  input  using  the  mapping  encoded  in  the  layer  during  training. 
The  input  and  output  feature  vectors  are  then  compared  and  the  RMS  prediction  error  between  the  two 
is  calculated.  A  large  RMS  error  implies  that  the  feature  vector,  for  whatever  reason,  is  novel  or  differ¬ 
ent  than  the  class  of  data  that  the  network  has  been  trained  on.  Note  that  the  network  does  not  classify 
this  vector  to  a  specific  class  of  data,  it  just  denotes  the  feature  vector  as  different  or  not  a  member  of 
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different  or  not  a  member  of  the  class  of  data  that  the  network  was  trained  on.  Essentially,  the  net¬ 
work  will  tell  you  when  something  is  novel  not  why  it  is  novel. 

In  the  AVDS  gearbox  application,  the  AA  network  had  been  trained  using  the  original  (including 
the  in-flight  data  from  the  four  good  A/C)  RBF  training  set  of  no-defect  data.  The  test-rig  defect  data 
had  then  been  used  to  cross  validate  the  network  and  verify  the  threshold  setting  for  RMS  prediction 
error.  The  original  cross-valation  data  set  that  caused  the  RBF  network  to  indicate  an  anomaly  was 
then  presented  to  the  AA  network.  The  results  of  this  test  are  shown  in  figure  16. 
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Figure  16.  Blind  testing  RMS  prediction  error  vs.  feature  vector  run  for  the  trained  AA  network.  The 
feature  vectors  from  A/C  Bu  No.  156432  are  shown  in  black  while  the  feature  vectors  from  the  three 
other  A/C  in  the  cross-valation  test  set  are  shown  in  white.  The  prediction  error  threshold  is  set  at 
approximately  0.75. 


The  figure  shows  that  AA  net  correctly  indicated  the  anomaly  and  did  not  false  alarm  for  the  three 
other  A/C  in  the  cross-valation  test  set.  The  AA  net  was  able  to  correctly  identify  an  anomalous 
condition  consisting  of  multiple  defects  in  an  A/C  when  trained  using  only  positive  no-defect  data. 
The  AA  net  also  demonstrated  the  capability  to  correctly  assess  the  condition  of  multiple  A/C  with 
which  the  network  had  no  previous  experience. 
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6  NEURAL  NETWORK  PROCESSOR  (N3P)  FLIGHT  TRIALS 

6.1  INTRODUCTION 

The  AVDS  CH-46E  Flight  Trials  demonstrated  the  potential  of  the  N3P  q)proach  to  trans¬ 
mission  diagnosis.  During  the  course  of  the  trials,  there  were  no  false  alarms  and  all  classifica¬ 
tions  of  the  status  of  the  ^  main  transmission  of  A/C  692  were  to  a  healdiy  transmission.  All 
rlqgdfiraiinTig  were  made  by  die  N3P  without  having  to  baseline  the  system  to  the  transmission 
of  A/C  692.  The  potential  of  the  approach  to  identify  those  situations  where  sensor/data  flaws 
bring  into  question  the  rehability  of  the  results  was  also  demonstrated.  Unfortunately,  the 
minimal  quantify  of  data  collected  from  a  single  A/C  during  the  trials  only  shows  the  potential 
of  the  approach  and  does  not  provide  a  critical,  defensible,  quantified  metric  on  the  perform¬ 
ance  of  the  system. 

For  the  analysis  of  the  flight  trials,  it  will  be  of  benefit  to  quickly  review  the  installation  and 
communications  of  the  N3P  systems.  The  N3P  hardware  was  installed  in  an  avionics-hardened 
VMEbus  chassis  for  placement  in  the  A/C.  A  Teledyne-supplied  signal  conditioning  card,  a 
Teledyne-supphed  phase  lock  loop  and  oscillator  card,  and  a  fli^t  regime  processor/com¬ 
munications  card  were  also  installed  in  the  VMEbus  chassis  (tiie  installation  is  shown  in  figures 
1 7  and  1 8).  The  N3P  would  receive  analog  conditioned  signals  from  the  aft  main  transmission 
accelerometers  and  an  analog  conditioned  rotor  position  shaft  tachometer  signal  via  the  Tele- 
dyne-suppUed  cards.  The  N3P  would  also  receive  Data  Acquisition  Fhght  Conditions  (DAFC) 
from  the  MDAU  via  the  flight  regime  processor  card  at  a  4-Hz  rate  for  use  in  data  processing. 
In  turn,  the  N3P  would  provide  a  status  to  the  MDAU  at  a  1  -Hz  rate.  Note  that  the  health 
status  of  the  transmission  is  not  necessarily  calculated  at  a  1-Hz  rate.  In  the  case  where  all 
conditions  were  present  so  that  the  N3P  could  analyze  and  classify  the  condition  of  the  trans¬ 
mission,  this  calculation  took  approximately  3  seconds.  The  intervening  status  messages  that 
went  to  the  MDAU  would  be  latched  to  the  previous  status  in  the  oufyut  buffer  during  the  cal¬ 
culation.  Thus,  the  total  number  of  status  messages  does  not  necessarily  represent  on  a  one-  to- 
one  basis  the  amount  of  data  (in  seconds)  that  was  processed  by  the  system. 
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Figure  17.  Front  view  of  the  VMEbus  installation  in  A/C  692.  The  VMEbus  chassis  contain¬ 
ing  the  N3P  is  located  in  the  lower  portion  of  the  equipment  rack.  The  OQAR  that  logged  the 
N3P  status  message  is  the  rectangular  black  box  shown  in  the  upper  left  of  the  equipment 


Figure  1 8.  Side  view  of  the  VMEbus  installation. 

6.2  PROCESSING  THE  RESULTS  OF  THE  TRIAL 

Once  tiie  N3P  status  message  was  received  by  the  MDAU  via  the  flight  regime  processor 
card,  it  was  then  sent  to  die  OQAR  for  data  logging.  The  status  message  consisted  of  four 
fields;  the  report  indicator,  fault  location,  fault  severity,  and  DAFC.  A  report  indicator  of  value 
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1  to  3  indicates  a  healthy  transmission  (for  a  breakdown  of  the  possible  report  indicators,  please 
see  table  12).  A  report  indicator  of  4  to  8  indicates  a  faulty  or  anomalous  transmission. 


Table  12.  Report  indicator  breakdown. 


Report  Indicator  Value 

Description 

0 

No  Report  At  This  Time 

1 

Good  (Radial  Basis  Function) 

2 

Good  (Auto-Associator) 

3 

Good  (Radial  Basis  Function  and  Auto- 
Associator) 

4 

Anomaly  (Auto-Associator) 

5 

Anomaly  (Radial  Basis  Function  and  Auto- 
Associator) 

6 

Anomaly  &  Fault 

7 

Fault  (Radial  Basis  Function) 

8 

Fault  (Radial  Basis  Function  and  Auto- 
Associator) 

9 

Neural  Network  Confidence  Failure 

The  fault  location  and  fault  severity  fields  were  never  used  during  the  trials  since  no  faults  or 
anomalies  were  indicated  during  the  course  of  the  trials.  The  DAFCs  in  which  the  N3P  at¬ 
tempted  to  process  data  and  calcialate  the  status  of  the  transmission  are  listed  in  table  13.  Note 
that  die  DAFC  had  to  have  been  stable  for  approximately  15  seconds  prior  to  and  approxi¬ 
mately  1 5  seconds  after  the  calcidation  in  order  for  the  health  status  of  the  transmission  to  be 
output  from  the  N3P  to  the  MDAU. 

After  the  flight,  the  N3P  status  messages  on  tihe  OQAR  disc  could  be  browsed  and  con¬ 
verted  for  analysis  using  the  Teledyne-supplied  FLIDRAS  software.  The  converted  files  then 
had  to  be  edited  prior  to  importation  into  the  N3P  groundstation  for  display.  During  and  after 
the  trials,  the  process  of  downloading  the  N3P  messages  turned  out  to  be  a  convoluted  and 
time  consuming  process  that  increased  the  time  latency  of  the  results  and  made  troubleshooting 
and  analysis  a  difficult  and  tiresome  process. 

In  the  N3P  groundstation,  die  analysis  of  the  results  was  limited  to  a  sequence  of  histogram 
displays  used  to  analyze  the  results  and  an  overall  status  of  the  aft  main  transmission  and  N3P 
system  for  the  flight.  The  messages  could  be  broken  down  to  display  histograms  of  die  status 
message  sorted  by  types  of  report  indicator  (figure  19),  confidence  indicated  in  the  answer 
(figure  20),  and  subsets  of  the  answer  corresponding  to  no  fault  and  faults/anomaly. 
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Histograms  of  the  DAFCs  received  from  the  flight  Regime  Processor  could  also  be  dis¬ 
played  in  the  groundstation.  The  interim  message  file  could  also  be  manually  examined  in 
the  groundstation  to  develop  trend  information  and  to  select  subsets  of  the  flight  for  analysis 
and  display. 


Table  13.  DAFC  breakdown. 


DAFC  Value 

Description 

0 

Flat  pitch  (27%  torque) 

1 

Light  on  wheels  (50%  torque) 

2 

Hover  in  ground  effect  (70%  torque) 

3 

Hover  out  of  ground  effect  (75%  torque) 

4 

750  feet  per  minute  descent  (30%  torque) 

5 

60  knots  indicated  air  speed  (45%  torque) 

6 

80  knots  indicated  air  speed  (40%  torque) 

7 

100  knots  indicated  air  speed  (60%  torque) 

8 

120  knots  indicated  air  speed  (80%  torque) 

9 

140  knots  indicated  air  speed  (100% 
torque) 

10 

• 

Spare  (not  currently  used) 

• 

13 

Spare  (not  currently  used) 

14 

Unknovm  on  ground 

15 

Unknown  in  flight 
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N3P  Report  Histogram 

Flight  Name  =  Flt-6  Record  Count  =  6137 


Figure  19.  Sample  histogram  from  Flight  N3SHA-6  showing  distribution  report  indicators 
broken  down  into  fields  of  good  transmission,  anomalous-faulty  transmission,  no  report  at 
this  time,  and  network  confidence  failures.  During  this  flight,  multiple  sensor/wiring  failures 
related  to  loose  connectors  and  an  accelerometer  backing  off  its  mounting  bolt  caused  net¬ 
work  confidence  failures. 


N3P  NN  Confidence  Tabulation 

Flight  Name  =  Flt-6  Record  Count  =  6137 


1 


Report  Indicator  Value  Groups 

Figure  20.  Sample  histogram  from  Flight  N3SHA-6  showing  confidence  indicated  in  the  re¬ 
port  indicators  (figure  19).  In  this  case,  the  majority  of  the  no-fault  answers  provided  by  the 
RBF  networks  had  marginal  confidence  while  all  answers  provided  by  the  Auto-Associator 
networks  and  some  of  the  answers  by  the  RBF  networks  had  a  relatively  low  confidence. 
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6.3  PROCESSED  FLIGHT  TRIALS  DATA 

During  the  course  of  the  trials,  there  were  18  dedicated  flights  and  one  scripted  piggy  back 
flight  (the  flight  to  and  firom  the  USS  Saipan  LHA2).  For  the  first  flight,  the  N3P  system  had  a 
hardware  failure  that  prevented  the  system  from  booting  and  operating  during  the  flight.  The 
N3P  system  was  not  installed  aboard  the  A/C  during  the  second  flight  due  to  troubleshooting  of 
the  hardware.  For  the  remaining  17  flights,  at  least  some  data  from  each  flight  are  available  for 
analysis  (table  14).  It  is  worth  emphasizing  that  these  data  were  derived  without  baselining  the 
system  to  the  transmission  of  692,  and  that  the  N3P  software  operated  witiiout  flaw  during  the 
trials.  Two  software  changes  were  made  dming  the  trials,  aroimd  Flight  N3SHA-13,  in  an  at¬ 
tempt  to  aid  in  troubleshooting  the  DAFC  calculation  and  sensor  failures,  but  these  changes  did 
not  affect  the  diagnostic  or  performance  characteristics  of  the  N3P.  One  software  change  al¬ 
lowed  time  code  to  be  written  to  exabyte  tqjes  to  help  in  troubleshooting  the  DAFC  calcula¬ 
tions  while  the  other  change  stored  an  output  file  on  the  hard  disk  to  allow  "qitick  look"  trou¬ 
bleshooting  of  the  sensors  and  wiring. 


Table  14.  Report  indicators  broken  down  by  flight.  Flight  N3SHA6-8  was  particularly 
plagued  with  multiple  sensor  failures  including  bad  microdot  connections,  accelerometers 
backing  off  mounts,  problematic  patch  panel  connections,  intermittent  signal  saturation,  etc. 


FLIGHT 

No  Report  Indicator 

Good  Indicator 

Confidence  Failure 

Anomaly/Fault 

N3SHA-3 

5993 

129 

0 

0 

N3SHA-4 

3864 

240 

0 

0 

N3SHA-5 

3356 

234 

0 

0 

N3SHA-6 

5831 

144 

162 

0 

N3SHA-7 

2060 

162 

138 

0 

N3SHA  -8 

5806 

114 

60 

0 

N3SHA  -9 

5282 

570 

0 

0 

N3SHA-‘10 

5947 

779 

0 

0 

N3SHA  -11 

5375 

479 

0 

0 

N3SHA  -12 

4631 

334 

0 

0 

N3SHA  -13 

5190 

996 

0 

0 

N3SHA  -14 

1970 

381 

0 

0 

N3SHA  -15 

6708 

381 

0 

0 

N3SHA  -16 

6157 

144 

0 

0 

N3SHA  -17 

5429 

415 

0 

0 

N3SHA  -18 

2525 

807 

0 

0 

N3SHA  Saipan 

4400 

162 

0 

0 

Total 

80524 

6471 

360 

0 

The  majority  of  no  report  at  this  time  indicators  encountered  during  the  trials  were  due  to  de¬ 
velopment  and  refinement  of  the  DAFC  calculation  and  limits.  Many  of  the  remaining  no  report 
indicators  and  the  network  confidence  failure  indicators  are  due  to  sensor  failures,  connector 
failures,  cabling  failures,  and  equipment  failures.  Under  the  operating  conditions  of  the  trials, 
the  A/C  spent  a  fair  amount  of  time  transitioning  between  DAFCs  and  in-flight  regimes  that 
were  not  covered  by  the  DAFCs  used  by  the  N3P.  This  normal  operation  accounts  for  the  re¬ 
mainder  of  the  no  report  indicators. 
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From  the  above  data,  two  trends  are  important  to  note.  First,  over  the  course  of  the  trials  the 
DAFC  calculations  did  tend  to  improve  with  a  corresponding  rise  in  the  number  of  report  indi¬ 
cators  that  actually  classified  the  health  of  the  transmission.  This  trend  is  positive,  but  not  mo¬ 
notonic.  Secondly,  right  up  to  the  end  of  the  trials  the  installation  was  plagued  by  sensor  and 
cabling  associated  failures.  There  were  also  several  unscripted  piggyback  flights  where  the 
N3P  was  installed  aboard  the  A/C,  but  due  to  DAFC  troubleshooting  and  faulty  SCSI  drive 
cabling,  there  is  no  N3P  data  available  for  analysis  from  these  flights. 

6.4  FLIGHT  TRIALS  CONCLUSIONS 

During  the  course  of  the  trials,  the  N3P  system  was  up  and  running  for  a  total  of  24.3  hours. 
The  N3P  system  provided  1.9  hours  of  status  reports  containing  a  condition  identification  other 
than  no  report  at  a  1 -second  interval  rate.  Of  these  1.9  hours,  1.8  hours  were  actual  classifica¬ 
tions  as  to  the  health  of  the  transmission.  This  1.8  hours  of  no  fault  transmission  classifications 
corresponds  to  7.8%  of  the  time  that  the  N3P  was  up  and  running.  The  actual  amount  of  data 
that  were  used  to  classify  the  transmission  as  to  a  no-fault  status  for  this  7.8%  of  N3P  uptime 
was  0.60  hours  or  36  minutes. 

There  were  no  false  alarms  recorded  during  flie  course  of  the  trials.  The  total  amount  of  time 
that  comprised  an  opportunity  for  a  false  alarm  was  1.9  hours.  The  underlying  amount  of  data 
comprising  this  opportunity  correspond  to  0.63  hours  or  37.8  minutes.  This  window  of  oppor¬ 
tunity  for  false  alarm  corresponds  to  2.6%  of  the  total  N3P  uptime.  It  should  be  obvious  that 
extraction  of  a  meaningful  false  alarm  rate  from  this  minimal  amount  of  data  from  a  statistical 
sample  of  one  transmission  is  not  possible.  Therefore,  even  though  no  false  alarms  were  indi¬ 
cated  during  the  trials  this  is  not  taken  to  indicate  with  any  confidence  that  the  N3P  false  alarm 
rate  is  zero.  The  final  conclusion  is  that  the  trials  were  encouraging,  but  by  no  means  a  defini¬ 
tive  measure  of  the  performance  of  the  N3P  system. 
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7  PROGRAM  CONCLUSIONS 

The  capability  of  the  N3P  system  to  perform  diagnostics  without  baselining  to  a  particular 
build  of  die  given  transmission  was  demonstrated  in  all  three  phases  of  the  program.  During 
the  Test  Stand  portion  of  the  program  the  neural  networks  were  trained  using  data  from  one 
build  of  the  transmission  and  blind-tested  on  data  from  a  build  separated  by  nine  major  mainte¬ 
nance  actions.  The  networks  successfully  classified  the  "blind-build"  as  no  fault.  When  the 
system  was  trained  using  in-flight  data,  only  data  from  four  of  the  available  A/C  were  used 
during  training.  The  data  from  the  remaining  four  A/C  were  used  to  blind-test  the  system. 

There  was  no  baselining  of  the  system  to  this  set  of  data.  It  was  out  of  this  set  of  data  that  the 
find  in  AJC  1 56432  came.  As  previously  noted,  the  system  had  an  anomalous  response  to  A/C 
1 56432  and  successfully  classified  flie  other  A/C  as  no  fault.  During  the  flight  trials,  the  system 
was  installed  on  A/C  692  without  baselining.  The  system  was  able  to  correctly  diagnose  the 
condition  of  the  A/C  without  using  data  from  the  A/C  to  "tune"  flie  system  to  this  A/C. 

Being  able  to  operate  the  system  without  baselining  to  a  given  A/C  is  a  significant  achieve¬ 
ment  and  a  valid  goal  for  the  program.  Typically,  it  takes  COTS  systems  on  the  order  of  20 
dedicated  flight  hours  to  baseline  to  a  given  A/C.  This  baselining  must  be  performed  for  each 
system  installed  on  every  A/C.  The  baselining  must  be  repeated  if  a  major  maintenance  action 
is  performed  on  the  transmission  of  the  A/C.  As  one  can  see,  the  baselining  effort  is  by  no 
means  trivial  for  a  significant  fleet  of  A/C.  In  particular,  the  Marine  Corps  alone  has  on  flie  or¬ 
der  of  238  CH-46E  A/C  which  would  require  ^proximately  4760  flight  hours  of  dedicated 
baselining. 

The  N3P  system  has  also  been  demonstrated  in  all  three  phases  of  the  program  to  operate 
across  a  range  of  torques.  For  all  phases,  there  were  at  least  nine  torque  settings  where  the  N3P 
was  consistently  shown  to  operate  and  assess  the  condition  of  the  transmission.  This  is  also  a 
valid  goal  for  the  program  since  in  a  military  environment  the  A/C  are  operated  in  a  manner  of 
conditions  across  a  wide  range  of  regimes.  A  diagnostic  system  should  adapt  itself  to  the  man¬ 
ner  in  which  flie  A/C  will  be  flown  since  military  flight  operations  can  hardly  be  expected  to 
ad^t  themselves  to  the  requirements  of  a  diagnostic  system. 

7.1  RECOMMENDATIONS 

hi  light  of  flie  eiqierience  gained  during  the  course  of  the  program,  the  following  recommen¬ 
dations  are  made; 

•  Accelerometers 

The  microdot  coimectors  of  the  Endevco  7259A-10  accelerometers  proved  to  be  quite 
problematic  during  the  inflight  phase  of  the  program  and  the  course  of  the  flight  trials. 
There  is  a  major  reliability  versus  sensitivity/bandwidfli  tradeoff  issue  that  should  no  longer 
be  a  concern  with  many  of  the  accelerometers  available  today.  Most  cinrent  accelerometers 
have  a  more  than  sufficient  sensitivity  and  bandwidth  for  diagnostic  purposes  that  makes 
the  tradeoff  issue  become  one  of  reliability.  Accelerometers  should  be  chosen  first  on  the 
basis  of  their  rehabihty  and  durability  in  the  helicopter  avionics  environment,  with  sensitiv¬ 
ity  and  bandwidth  secondary  issues.  It  is  also  recommended  that  the  signal  pafli  from  the 
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accelerometers  to  the  diagnostic  system  be  made  as  simple  as  possible  with  as  few  intervening 
connectors  as  possible. 

•  Accelerometer  Mounts 

The  accelerometer  mounts  should  have  their  vibrational  response  characterized  prior  to  use.  If 
the  response  has  resonances  or  characteristics  that  would  tend  to  mask  the  vibration  signature,  an 
alternate  mount  should  be  designed  and  characterized. 

•  Accelerometer  Locations 

The  location  of  the  accelerometers  should  be  chosen  to  maximize  the  visibility  of  the  compo- 
nent(s)  that  the  accelerometer  is  chosen  to  cover.  Interference  in  the  signal  from  other  compo¬ 
nents  and  transmission  case  resonances  should  be  minimized.  As  an  example  of  signal  interfer¬ 
ence,  the  AVDS  accelerometer  #8  signal  was  dominated  by  the  vibration  from  the  underbody  oil 
scavenge  pump.  The  signals  from  gear  box  accessories  and  other  components  were  effectively 
masked  by  the  pump  signature.  To  adequately  determine  locations  that  maximize  the  visibility  of 
monitored  components,  the  accelerometer  locations  should  be  determined  using  an  instrumented 
survey. 

•  Data  Quality  Assurance 

On-line,  automated  data  quality  assurance  should  be  available.  This  data  quality  assurance 
should  be  both  passive  and  active.  The  types  of  quality  assurance  should  be  chosen  to  ensure  that 
the  data  is  free  from  defects  and  artifacts,  but  the  implementation  should  be  chosen  so  that  ease 
of  use  and  availability  for  real-time  troubleshooting  are  maximized. 

•  Flight  Trials  Performance  Metrics 

The  performance  metrics  used  to  assess  performance  should  be  well-defined  and  chosen  in  ad¬ 
vance  of  the  trials.  The  metrics  should  be  unambiguous,  easy  to  compute,  and  adequately  charac¬ 
terize  any  performance  tradeoffs  related  to  the  system.  The  metrics  should  also  provide  an  ap- 
ples-to-apples  comparison  between  different  systems  that  are  being  evaluated  at  the  same  time. 
This  is  especially  true  in  the  case  where  false  alarm  rates  are  a  significant  measure  of  the  per¬ 
formance  of  the  system.  In  particular,  many  COTS  systems  have  a  decision  interval  on  a  given 
component  that  is  on  the  order  of  minutes.  By  contrast,  the  N3P  has  a  decision  interval  on  the 
complete  assembly  on  the  order  of  3  seconds.  In  order  to  compensate  for  the  differing  decision 
intervals  and  component  coverages,  the  false  alarm  rates  calculated  for  the  two  types  of  systems 
would  have  to  be  time  base  normalized  and  component  coverage  normalized. 

•  Flight  Trials  Design 

The  design  of  the  flight  trials  should  be  driven  by  the  performance  metrics  used  to  quantify 
performance  during  the  trials.  The  trials  should  be  designed  so  that  the  metrics  can  be  established 
and  properly  quantified.  Design  of  the  trials  should  also  place  the  different  systems  under 
evaluation  on  an  equal  footing  during  the  trials. 
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•  Representative  Fleet  Sample 

In  order  for  the  results  of  the  trials  to  be  statistically  meaningful,  the  trials  should  encompass  a 
greater  period  of  time  spread  across  a  larger  number  of  A/C.  On  each  A/C,  the  systems  should 
be  evaluated  for  approximately  10%  of  the  life  of  the  transmission  between  rebuilds.  The 
evaluation  should  take  place  on  a  representative  sample  of  the  A/C  in  the  fleet.  Approximately 
5-10%  of  the  A/C  comprising  the  fleet  should  have  systems  installed  and  evaluated  during  the 
course  of  the  trials. 

•  Continued  Development  of  Neural  Network  Diagnostic  Techniques 

Test  Stand,  In-flight,  and  Flight  Trials  results  suggest  that  the  N3P  approach  appears  to  offer 
significant  advantages  over  COTS  technology,  but  that  the  approach  requires  further  evaluation. 
Since  the  results  to  date  have  been  so  promising,  continued  development  of  the  N3P  system 
should  be  pursued.  The  system  should  be  evaluated  on  a  number  of  fleet  A/C  for  a  longer  dura¬ 
tion  so  that  credible  false  alarm  rate  statistics  can  be  established.  This  evaluation  should  take 
place  prior  to  efforts  to  "productionize"  the  system. 
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APPENDIX  A 

AIR  VEHICLE  DIAGNOSTIC  SYSTEM  SIGNAL  PROCESSING  AND 
CLASSIFICATION  ROUTINES.  BOARD-LEVEL  IMPLEMENTATION. 
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